variability-aware parsing -- oopsla talk
DESCRIPTION
OOPSLA 2011 talk about variability-aware parsing (i.e., parsing C code without preprocessing it first) Corresponding paper: http://www.informatik.uni-marburg.de/~kaestner/oopsla11_typechef.pdfTRANSCRIPT
Variability-Aware Parsingin the Presence of
Lexical Macros and Conditional Compilation
Christian Kästner
Paolo G. Giarrusso
Tillmann Rendel
Sebastian Erdweg
Klaus Ostermann
Thorsten Berger
Parsing C Codewithout Preprocessing
Christian Kästner
Paolo G. Giarrusso
Tillmann Rendel
Sebastian Erdweg
Klaus Ostermann
Thorsten Berger
Viewer Discretion Is Advised
LinuxKernel
10,000 features, 6 million lines of C code
Parse and Type checkall configurations of the
entire Linux kernel
10,000 features, 6 million lines of C code
AST with
Variability Informationtrue -> (WORLD v BYE)
¬ (WORLD ˄BYE)
greet.c
printf
msg
VWORLD VBYEprintf
msg msg
main
ε ε
…
Variability-Aware AnalysisType SystemStatic AnalysisBug FindingTestingModel CheckingTheorem Proving…
Parsing Cwithout Preprocessing
greet.c
printf
msg
VWORLD VBYE+ printf
+ msg + msg
+ main
ε ε
???
Macro expansion needed for parsing Alternative macros
Undisciplined annotations
Previous Solutions
Disciplined SubsetRequires Code Preparation
Heuristics and Partial AnalysisInaccurate, False Positives
Brute ForceInfeasible Effort
TypeChef
Variability-Aware
Parser-Framework
+
* VA
2 43 5
Variability-Aware
Parser
Variability-Aware
Analysis
https://github.com/ckaestne/TypeChef
Variability-Aware
Lexer
(2
*3
)+
4A5¬A
Variability-Aware Lexer
(2
*3
)
Variability-Aware Lexer
0 included header files per C file
0 macros per C file
0 conditional macros per C file
0 token per C file
0 % conditional
35385901387
33549072
X862.6.33.3
2 * 3 + 4A 5¬A
+
* VA
2 43 5
Variability-Aware Parsing
( )
Undisciplined Annotations
* (A 3 4 )A
+
*
VA
2
4
3
2 +
*
+
3
2
4
Partial Errors
* (A 3 4 )
Parse Error
VA2 +
*
+
3
2
4
Expected + or EOF, but found ) at 1:7
3 + 4A 4¬A B˄ +¬A B˄ 6¬A( (¬A )¬A )
true
3 + 4A 4¬A B˄ +¬A B˄ 6¬A( (¬A )¬A )
true
3 + 4A 4¬A B˄ +¬A B˄ 6¬A( (¬A )¬A )
true3
3 + 4A 4¬A B˄ +¬A B˄ 6¬A( (¬A )¬A )
true3
3 +
4A 4¬A B˄ +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
3
3 +
4A 4¬A B˄ +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
3
3 +
4A 4¬A B˄ +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
3
Next Token:
ctx -> tok: Consumectx ->¬tok: Skipelse: Split (ctx ˄pc) and (ctx ˄ ¬pc)
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
3
3 +
4A 4¬A˄B
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
+¬A B˄ 6¬A3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )¬A4¬A B˄ +¬A B˄ 6¬A )
¬A
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A
)¬A )
¬A˄¬B
4¬A B˄ +¬A B˄ 6¬A
4¬A B˄ +¬A B˄ 6¬A )¬A )
¬A˄B
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A
)¬A )
¬A˄¬B
4
4¬A B˄ +¬A B˄ 6¬A
4¬A B˄ +¬A B˄ 6¬A )¬A )
¬A˄B
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A
)¬A )
¬A˄¬B
4
4¬A B˄ +¬A B˄ 6¬A
4¬A B˄ +¬A B˄ 6¬A )¬A )
¬A˄B
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A
)¬A )
¬A˄¬B
4¬A B˄ +¬A B˄ 6¬A
4¬A B˄ +¬A B˄ 6¬A )¬A )
¬A˄B
+
64
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A
)¬A )
¬A˄¬B
4¬A B˄ +¬A B˄ 6¬A
4¬A˄B +¬A B˄ 6¬A )¬A )
¬A˄B
4
3 +
64
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A
)¬A )
¬A˄¬B
4¬A B˄ +¬A B˄ 6¬A
4¬A˄B +¬A B˄ 6¬A )¬A )
¬A˄B
4
3 +
64
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A
)¬A )
¬A˄¬B
4¬A B˄ +¬A B˄ 6¬A
4¬A˄B +¬A B˄ 6¬A )¬A )
¬A˄B
6
4
3 +
64
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )4¬A B˄ +¬A B˄ 6¬A
4¬A˄B +¬A B˄ 6¬A
)¬A
¬A
4
+
64
VB
6
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A )
A
4A (¬A )4¬A B˄ +¬A B˄ 6¬A
4¬A˄B +¬A B˄ 6¬A
)¬A
¬A
+
64
VB
6
4
3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A
4A (¬A
4¬A B˄ +¬A B˄ 6¬A
4¬A˄B +¬A B˄ 6¬A
)¬A
+
64
VB
6
)
4
VA
true3
3 +
4A 4¬A˄B +¬A B˄ 6¬A
(
(¬A )¬A
4A (¬A
4¬A B˄ +¬A B˄ 6¬A
4¬A˄B +¬A B˄ 6¬A
)¬A
+
64
VB
6
)
true
4
VA
+
3
Variability-Aware
Parser Combinator Libraryfor Scala
Variability-Aware Parsers
GNU C + cpp Java + Antenna
X862.6.33.3
Parsing Linux
Variability in Build System (kbuild)and with #ifdef
Variability Model (kconfig)
0 C files (x86)
0 seconds per file (median)
0 hours total
0 % overhead (undiscipl.)
0
0 syntax errors
76653085
4.1
0X862.6.33.3
Variability-AwareParserType SystemModule SystemStatic AnalysisBug FindingTestingModel CheckingTheorem ProvingEditor SupportCode Transform.…
Variability-Aware Parsingin the Presence of
Lexical Macros and Conditional Compilation
Variability-Aware
Parser-Framework
+
* VA
2 43 5
Variability-Aware
Parser
https://github.com/ckaestne/TypeChef
Variability-Aware
Lexer
(2
*3
)+
4A5¬A
(macros, includes) (split & join)