boinc application development lifecycle
DESCRIPTION
BOINC Application Development Lifecycle. Rom Walton [email protected] September 10, 2008. BOINC Application Lifecycle. Developing. Validating. Debugging. Deploying. Developing Apps. Worker applications should call: Graphics applications should call: - PowerPoint PPT PresentationTRANSCRIPT
![Page 2: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/2.jpg)
BOINC Application Lifecycle
•Designing Application•Writing Code•Instrumenting Code•Fixing Bugs
•Running Test Workunits•Checking for Memory Leaks•Checking for Handle Leaks
•Checking Crash Reports•Formulating a Plan of Action
•Uploading Application•Uploading Support Files•Running update_versons
![Page 3: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/3.jpg)
Developing AppsWorker applications should call:
Graphics applications should call:
Benefits of the BOINC Diagnostic FrameworkStack traces on crashes (Win, Linux, Mac)Prevents the JIT Debugger dialog (Win)Memory leak detection (Win – VS Debug Builds)Heap corruption detection (Win – VS Debug
Builds)
boinc_init_diagnostics( BOINC_DIAG_DEFAULTS );
boinc_init_graphics_diagnostics( BOINC_DIAG_DEFAULTS );
![Page 4: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/4.jpg)
Developing Apps (cont’d)Platform neutral tips
Use a separate project to test your applicationsWindows tips
Use the symbol store technology, if possibleLinux tips
Link application with the BOINC-Build-Compat Virtual Machine. Application binds to older glibc but still contains the optimizations of newer compilers
Mac tipsBuild against the Mac OS X 10.3.9 SDK
![Page 5: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/5.jpg)
Developing Apps (cont’d)Windows Symbol Stores
What are they?How do they work?How does the BOINC Framework use them?How do I configure my Windows machine?
Set environment variable _NT_SYMBOL_PATH to:srv*c:\windows\symbols*http://msdl.microsoft .com/download/symbols
Execute:symchk c:\windows\system32\*.dll /ov
![Page 6: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/6.jpg)
Enable Symbols Demo
![Page 7: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/7.jpg)
Developing HighlightsUse the BOINC Diagnostic Framework
Traps structured exceptions (Win)Traps signals (Linux, Mac)Detects memory leaks (Win)Detects heap corruption (Win)
![Page 8: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/8.jpg)
BOINC Application Lifecycle
•Designing Application•Writing Code•Instrumenting Code•Fixing Bugs
•Running Test Workunits•Checking for Memory Leaks•Checking for Handle Leaks
•Checking Crash Reports•Formulating a Plan of Action
•Uploading Application•Uploading Support Files•Running update_versons
![Page 9: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/9.jpg)
Validating AppsRun through your test workunitsCheck for buffer overruns
Win: BOINC_DIAG_HEAPCHECKENABLEDLinux: Valgrind
Check for memory leaksWin:
BOINC_DIAG_MEMORYLEAKCHECKENABLEDLinux: Valgrind
Check for handle leaksWin: Process Explorer, ntsd (!handles), oh.exeLinux: lsof
![Page 10: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/10.jpg)
Validating Apps (cont’d)Global Flags (gflags.exe)
Not useful for debugging inthe field
Tweaks the heap manager(works with any type of application)
Only detects buffer overruns for dynamically allocated memory.
gflags.exe options:
![Page 11: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/11.jpg)
Validating Apps (cont’d)Example memory leak output:
Example handle leak output (oh.exe):
Dumping objects ->..\lib\diagnostics_win.C(642) : {70} normal block at 0x017C3A30, 1068 bytes long. Data: < T > 1C 1A 00 00 54 00 00 00 01 00 00 00 00 00 00 00 Object dump complete.
boinc.exe File 0008 \ProgramData\BOINC\stdoutdae.txtboinc.exe Key 0034 \REGISTRY\MACHINEboinc.exe File 0094 \ProgramData\BOINC\stderrdae.txt
![Page 12: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/12.jpg)
Validating HighlightsTest the following once per application
release cycle:Memory leaksHandle leaksHeap corruption
Test the following once per application release:Exception/Signal handling
(Abort a task from BOINC Manager)
![Page 13: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/13.jpg)
BOINC Application Lifecycle
•Designing Application•Writing Code•Instrumenting Code•Fixing Bugs
•Running Test Workunits•Checking for Memory Leaks•Checking for Handle Leaks
•Checking Crash Reports•Formulating a Plan of Action
•Uploading Application•Uploading Support Files•Running update_versons
![Page 14: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/14.jpg)
Deploying AppsAdd application to
‘$project_root/project.xml’.
$app_name = Whatever you want to call your application
Execute xadd to add your application to the project.
<boinc> <app> <name>$app_name</name> <user_friendly_name></user_friendly_name> </app></boinc>
cd $project_root./bin/xadd
![Page 15: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/15.jpg)
Deploying Apps (cont’d)Upload your application and its dependencies to:
$project_root/apps/$app_name/$app_ver_name$app_ver_name is the name of the worker
executableWorker executable name is defined as:
$name_$version_$platform
Execute update_versions t0 add app versions to project
Windows: uppercase_6.1_windows_intelx86.exeLinux: uppercase_6.1_i686-pc-linux-gnuMac: uppercase_6.1_i686-apple-darwin
cd $project_root./bin/update_versions
![Page 16: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/16.jpg)
Deployment Demo
![Page 17: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/17.jpg)
Deployment HighlightsApplication files are assumed to be immutableupdate_versions performs the following
functions:Copies application files to download directoryReads signature file/generates signature fileAdds application to databaseTouches feeder restart file
![Page 18: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/18.jpg)
BOINC Application Lifecycle
•Designing Application•Writing Code•Instrumenting Code•Fixing Bugs
•Running Test Workunits•Checking for Memory Leaks•Checking for Handle Leaks
•Checking Crash Reports•Formulating a Plan of Action
•Uploading Application•Uploading Support Files•Running update_versons
![Page 19: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/19.jpg)
Debugging Apps4% - 5% failure rates are not uncommon
Machines overheatBad memoryBad processors
Crash reports are stored inside the BOINC DatabaseGo to the project_ops web site to look at crash
reportsLook for reports that have:
PDB symbols for the application and librariesComplete stack traces
![Page 20: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/20.jpg)
Debugging Apps (cont’d)Determine environment is setup correctly
Check that your symbol store is listed
BOINC Windows Runtime Debugger Version 5.5.0
Dump Timestamp : 09/09/08 23:41:39Debugger Engine : 4.0.5.0Symbol Search Path: srv*C:\DOCUME~1\romw\LOCALS~1\Temp\symbols*http://boinc.berkeley.edu/alpha/symstore;srv*c:\windows\symbols*http://msdl.microsoft.com/download/symbols;srv*C:\DOCUME~1\romw\LOCALS~1\Temp\symbols*http://boinc.berkeley.edu/symstore
![Page 21: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/21.jpg)
Debugging Apps (cont’d)Determine that the loaded modules have
symbols
Symbols may not download Machine is off-line during crash Took longer that 30 seconds to download symbol
file(s)
ModLoad: 00400000 00060000 C:\...\uppercase_6.3_windows_intelx86.exe (PDB Symbols Loaded)ModLoad: 7c800000 000c0000 C:\...\ntdll.dll (5.2.3790.1830) (PDB Symbols Loaded)
![Page 22: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/22.jpg)
Debugging Apps (cont’d)Crashing thread contains an exception record:
Access Violation (0xC0000005) NULL references exceptions Using indexes as pointers Processor flipping bits because of heat
Breakpoint Encountered (0x80000003) User aborted an active task Assertion failure in debug builds
- Unhandled Exception Record -Reason: Breakpoint Encountered (0x80000003) at address 0x7C822583
![Page 23: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/23.jpg)
Debugging Apps (cont’d)Check call stack of crashing thread
Look for the interface point between your application and an OS library. Typically NTDLL.DLL or KERNEL32.DLL
- Callstack -ChildEBP RetAddr Args to Child00aafd60 00402221 00000000 00000000 00000000 00000001 ntdll!_DbgBreakPoint@0+0x000aaffb4 0042684e 77e66063 00000000 00000000 00000000 uppercase_5.10_windows_intelx86!worker+0x0 (c:\boincsrc\main\boinc\samples\uppercase\upper_case.c:181)
![Page 24: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/24.jpg)
Debugging Apps (cont’d)What information is contained in a stack frame
ChildEBP – isn’t really useful RetAddr – Return Address, should be within the
same address space as the previous calling frameArgs to Child – First four parameters to functionModule!FunctionSource File:Line Number
ChildEBP RetAddr Args to Child00aaffb4 0042684e 77e66063 00000000 00000000 00000000 uppercase_5.10_windows_intelx86!worker+0x0 (c:\boincsrc\main\boinc\samples\uppercase\upper_case.c:181)
![Page 25: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/25.jpg)
Debugging Apps (cont’d)When ‘X’ doesn’t mark the spot
Don’t get discouragedSample several crash reportsDebugging applications remotely can be hard
One nasty bug in the Rosetta@home applicationOnly happened during application destructionI looked at 120 crash reports over the course of 5
days before seeing the common themeResource contention between two threads
![Page 26: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/26.jpg)
Debugging Apps (cont’d)Need extra help?
Add trace statements by logging to stderrRelease a new version of your applicationOnly you know how your applications really
work
![Page 27: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/27.jpg)
Debugging HighlightsBOINC’s diagnostic framework
Processes crashes for youDumps relevant information to stderrNo need to manage dump or mini-dump filesNo need to understand the specifics of each
platforms infrastructureEasy to enable
![Page 28: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/28.jpg)
ConclusionBOINC Application Framework:
Mature diagnostic frameworkCompletely automatic feedback loopBOINC Client uses the same framework
We eat our own dog food
Should be compatible with all run-time environmentsC/C++FortranAnything that supports calling a C Function
![Page 29: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/29.jpg)
Questions and AnswersBOINC Website:
http://boinc.berkeley.edu/BOINC Development Mailing List:
[email protected] Projects Mailing List:
[email protected] Application Debugging:
http://boinc.berkeley.edu/trac/wiki/AppDebug
Any Questions?
![Page 30: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/30.jpg)
ToolsDebugging Tools for Windows (Windows)
gflags.exe, symchk.exe, symstore.exe, windbg.exeWeb site
Windows Resource Kit Tools (Windows)oh.exe, dh.exe, consume.exe, clear.exeWeb site
Process Explorer (Windows)DLL Dependencies, Handle Leak Detection, Real-
Time Thread Monitoring (Backtraces)Web site
![Page 31: BOINC Application Development Lifecycle](https://reader036.vdocuments.net/reader036/viewer/2022070407/5681439c550346895db0198f/html5/thumbnails/31.jpg)
Tools (cont’d)Valgrind (Linux)
Memory Leak Detection, Thread Error Detection
Web siteDDD (Linux)
Graphical DebuggerWeb site