Transcript
Page 1: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Joe FitzPatrick44Con 2014

Stupid PCIe Tricks

Page 2: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

● Electrical Engineering education with focus on CS and Infosec

● 8 years doing security research, speed debug, and tool development for CPUs

● Hardware Pen Testing of CPUs● Security training for functional validators

worldwide● Software Exploitation via Hardware

Exploits, AKA SExViaHEx

whoami

Joe FitzPatrick@securelyfitz

[email protected]

Page 3: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

If Joe Fitz...

Joe Sitz

Page 4: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

This is not academic-caliber research.

Lots of this stuff has been done before.

The difference is that I aim to show that PCIe attacks can be easy and cheap

Disclaimer

Page 5: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

What is PCIe?

Page 6: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

PCIe is PCI!

Page 7: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

PCIe is NOT PCI!

Foto tomada por Jorge González http://es.wikipedia.org

Photo by snikerdo http://en.wikipedia.org

Page 8: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Links and Lanes

Diagram: PCIe 2.1 specification

Page 9: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Hierarchy

Diagram: PCIe 2.1 specification

Page 10: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Switching and Routing

Diagram: PCIe 2.1 specification

Page 11: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Layers

Diagram: PCIe 2.1 specification

Page 12: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Configuration Space

Diagram: PCIe 2.1 specification

Page 13: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Configuration Space

Diagram: PCIe 2.1 specification

Page 14: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Configuration Space

Diagram: PCIe 2.1 specification

Page 15: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Configuration Space

Diagram: PCIe 2.1 specification

Page 16: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Configuration Space

Diagram: PCIe 2.1 specification

Page 17: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Enumeration

Diagram: PCIe 2.1 specification

Page 18: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Routing PCIe

Page 19: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

The Step-By-Step, Complicated, Mandatory, Inflexible Rules of Routing PCIe:

Page 20: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

1. route pairs adjacent and equal length

The Step-By-Step, Complicated, Mandatory, Inflexible Rules of Routing PCIe:

Page 21: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

1. route pairs adjacent and equal length

… that’s mostly it

The Step-By-Step, Complicated, Mandatory, Inflexible Rules of Routing PCIe:

Page 22: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Routing PCIeSystem Board Traces 12 Inches

Add-in Card Traces 3.5 inches

Chip-to-Chip Routes 15 inches

Follow these rules and your board might work.Break them and it might not.

Page 23: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Routing PCIe

Minimum PCIe:● 2.5GHz TX● 2.5GHz RX● 100MHz Clock (optional)

Page 24: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Routing PCIe

Cross-section of a USB 3.0 cable. Image courtesy of USB Implementers Forum

Page 25: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 26: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 27: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 28: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 29: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 30: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 31: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 32: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 33: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 34: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

A brief history of DMA attacks

Page 35: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Tribble

Page 36: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Firewire Attacks

Page 37: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 38: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 39: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 40: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Thunderbolt

Page 41: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Thunderbolt

Page 42: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

USB3380 Firmware

Page 43: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

USB3380 Firmware

> xxd SLOTSCREAMER.bin 0000000: 5a00 0c00 2310 4970 0000 0000 e414 bc16 Z...#.Ip........

Page 44: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

USB3380 Firmware

> xxd SLOTSCREAMER.bin 0000000: 5a00 0c00 2310 4970 0000 0000 e414 bc16 Z...#.Ip........

Page 45: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

USB3380 Firmware

> xxd SLOTSCREAMER.bin 0000000: 5a00 0c00 2310 4970 0000 0000 e414 bc16 Z...#.Ip........

That’s all!

Page 46: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

nsaplayset.org

Page 47: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Hardware

http://www.hwtools.net/PLX.html

Page 48: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Software

tools used in preparing this presentation:● plx’s flashing

software● pyusb + scripts● inception_pci● volatility for

memory analysis

Page 49: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Attack-side Software

Quick ‘n’ dirty PCIe memory read/write with PyUSB

Page 50: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

More attack-side Software

Page 51: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

More attack-side Software# EQUALS:## |-- Offset 0x00 # / # /\ |-patchoffset--------------->[b0 01] # 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f .. (byte offset)# -----------------------------------------------# c6 0f 85 a0 b8 00 00 b8 ab 05 03 ff ef 01 00 00 .. (chunk of memory data)# -----------------------------------------------# \______/ \___/ \______/# \ \ \# \ \ |-- Chunk 2 at internaloffset 0x05# \ |-- Some data (ignore, don't match this)# |-- Chunk 1 at internaloffset 0x00# \_____________________/# \# |-- Entire signature#

Page 52: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

More attack-side Software {'OS': 'Mac OS X 10.9', 'versions': ['10.9'], 'architectures': ['x64'], 'name': 'DirectoryService/OpenDirectory unlock/privilege escalation', 'notes': 'Overwrites the DoShadowHashAuth/ODRecordVerifyPassword return value. 'signatures': [{'offsets': [0x1e5], # 10.9 'chunks': [{'chunk': 0x4488e84883c4685b415c415d415e415f5d, 'internaloffset': 0x00, 'patch': 0x90b001, # nop; mov al,1; 'patchoffset': 0x00}]}]}]

Page 53: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Attacking via PCIe

Page 54: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick
Page 55: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

MRd

Find important values at known locations

Take memory dumps for later analysis

Example:Dump memory and use Volatility to analyze it

Page 56: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

DMA Stool Analysis with Volatility

dmesg log of the attack recovered from the memory dump of the victim

Page 57: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

DMA Stool Analysis with Volatility

names, pids, and uids for dumped processes

Page 58: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

DMA Stool Analysis with Volatility

extracted machine info

the perfect amount of memory to dump!

Page 59: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

MWr

Modify values at known locations

Manipulate code!!!

Example: Use Inception to modify lock screen checking, or drop a metasploit payload!

Page 60: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Inception with Metasploit (W7sp1 POC only)

Page 61: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

IORd/IOWr

Only for legacy devices

(legacy means not thoroughly tested recently)

Page 62: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

CfgRd/CfgWr

Interact with other PCI devices’ config spaces

Yet another separate address space/different means of accessing hardware

Page 63: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Msg/MsgD

Messages send things like interrupts and vendor-defined configuration

Many message types are very rarely used

Example: Johanna Rutkowska SNB VT-D stuff

Page 64: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Mitigations

Page 65: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Bus Master Enablejoefitz@linUX31a:~/Documents/pcie/SLOTSCREAMER/inception_pci$ lspci -vv | grep BusMaster

Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

Page 66: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Access Control Services

Page 67: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

IOMMU

Page 68: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Mitigating the Mitigations

Page 69: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

● Identifies device to the OS● OS chooses which driver to load● OS configures ACS, BME, etc…● OS loads driver

VID:PID

Page 70: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Default Drivers

● Some drivers are ‘class’ drivers (think USB MSC, etc...)

● Some device specific drivers might be installed by default (OSX)

● Drivers contain bugs● Think facedancer for PCIE or Thunderbolt

Page 71: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Early Boot

● IOMMU is not configured yet● Neither is much else● Wishlist: Volatility support for EFI shell

Page 72: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Option ROM/EFI drivers

● Some devices have firmware that gets run at early boot

● Some systems block this (but usually for anti-competitive reasons, not security)

Page 73: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Breaking the rules

● Spoof requesterID for posted transactions● Well-timed spoofed requesterID for non-

posted transactions● Setting the ‘translated request’ bit

Page 74: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Misconfigurations

● Everything is MMIO now - memory protections are essential

● Memory protections are not enough - need Cfg and IO protections as well - don’t forget about them

● Does installing a hypervisor change how your OS uses its IOMMU?

Page 75: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Putting it all togetherbecause 0.01% is too much

Page 76: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Thunderbolt

Diagram: Apple Thunderbolt Device Driver Programming Guide

Page 77: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

HALIBUTDUGOUT

Page 78: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Sorry, Previous Track 2 Speakers

ALLOYVIPER

Page 79: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 80: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 81: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 82: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 83: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 84: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 85: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 86: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Building ALLOYVIPER

Page 87: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

MITMing

Page 88: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

⇐ Thanks for the slides, snare & rzn

Page 89: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

⇐ Thanks for the slides, snare & rzn

Page 90: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Bypassing VT-d on Macbooks?● VT-d is off at boot/reboot● Broadcom Ethernet drivers crash the system● System reboots - all the doors are open for a

few moments

No POC yet (I’ll GTFO soon…)

Page 91: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Can we do it without imitating a device?

● Some PCIe switches have ‘transparent’ mode

● Some PCIe switches have TLP injection debug features

● Can we build one into a genuine device?● Can we build one into a cable?No POC yet here either

Page 92: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Potential enhancements

● 64-bit DMA (>4gb access!)● Full control over TLP Header

○ spoofing requester ID○ testing ‘reserved’ bits

Enough unproven concepts… time to GTFO...

Page 93: 44CON 2014 - Stupid PCIe Tricks, Joe Fitzpatrick

Joe FitzPatrick@securelyfitz

[email protected]://www.securinghardware.com

Questions?


Top Related