virtual separation of concerns (2011 update)
DESCRIPTION
Revised and extended version of the "Virtual Separation of Concerns" talk, prepared for the GI-Dissertationspreis colloquium at Dagstuhl and for the invited talk at Oregon State University in Corvallis.TRANSCRIPT
Virtual Separation of Concerns
Christian Kästner
Photo (cc) Horia Varlan
Feature-Oriented
Product Lines
DatabaseEngine
Printer Firmware
LinuxKernel
7
Variability = Complexity
a unique configuration for every
person on this planet
33 featuresoptional, independent
320 features
more configurations than estimated
atoms in the universe
optional, independent
Correctness?
Maintenance?Comprehension?
static int _rep_queue_filedone(...)DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return (__db_no_queue_am(dbenv));
#elsedb_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;
#ifdef DIAGNOSTICDB_MSGBUF mb;
#endif...
Excerpt from Oracle’s Berkeley DB
Conditional Compilation
Objections / Criticism
“#ifdef considered harmful”“#ifdef hell”
Designed in the 70th and hardly evolved since
“preprocessor diagnostics are poor”
“is difficult to determine if the code being viewed is actually compiled into the system”
“programming errors are easy to make and difficult to detect”
“incomprehensible source texts”
“maintenance becomes a ‘hit or miss’ process”
“CPP makes maintenance difficult”
“source code rapidly becomes a maze”
Academia: Compositional Approaches
class Stack { void push(Object o) { elementData[size++] = o; } ...}
class Stack { void push(Object o) { elementData[size++] = o; } ...}
refines class Stack { void push(Object o) { Lock l = lock(o); Super.push(o); l.unlock(); } ...}
refines class Stack { void push(Object o) { Lock l = lock(o); Super.push(o); l.unlock(); } ...}
Base / Platform
Feature: Queue
Feature: Diagnostic
aspect Diagnostics { ...}
aspect Diagnostics { ...}
class Stack { void push(Object o) { Lock l = lock(o); elementData[size++] = o; l.unlock(); } ...}
class Stack { void push(Object o) { Lock l = lock(o); elementData[size++] = o; l.unlock(); } ...}
Composition
ModuleComponentsFrameworks, Plug-insFeature-Modules / Mixin Layers / …Aspects / Subjects, Hyper/J, Deltas
A CLOSER LOOK AT PREPROCESSORS
Separation of Concerns
Obfuscationclass Stack {
void push(Object o#ifdef SYNC
, Transaction txn#endif
) {if (o==null
#ifdef SYNC|| txn==null
#endif) return;
#ifdef SYNCLock l=txn.lock(o);
#endifelementData[size++] = o;
#ifdef SYNCl.unlock();
#endiffireStackChanged();
}}
Femto OS
Obfuscation
Error-Pronestatic int _rep_queue_filedone(...)
DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return
(__db_no_queue_am(dbenv));#else
db_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;
#ifdef DIAGNOSTICDB_MSGBUF mb;
#endif// over 100 lines of add. code
}#endif
#ifdef TABLESclass Table { void insert(Object data, Txn txn) { storage.set(data, txn.getLock()); }}#endifclass Storage {#ifdef WRITE boolean set(…) { ... }#endif}
Advantages• Easy to use
• “annotate and remove”• Already part of many languages / simple tools• Most developers already familiar
• Flexible / Expressive• Language-independent / Uniform
• Low Adoption Barrier• esp. with legacy code
IMPROVING TOOL SUPPORT
Views
[ICSE’08, ViSPLE’08]
View on Feature Eval
View on Configuration “Add and Eval”
[ICSE’08, ICSE‘10, EASE‘11]
FeatureCommander
Controlled
Experiments
Scaling? How Used?
Faster?
3
ERROR DETECTION / PREVENTION
Disciplined
#ifdef WORLDint main() {
printf(“Hello World”);}#endifint main() {#ifdef WORLD
printf(“Hello World”);#endif}
Annotations
Align with code structure
greet.c
printf
msg
VWORLD VBYEprintf
msg msg
main
ε ε
Align with variability model
Disciplined
#ifdef WORLDint main() {
printf(“Hello World”);}#endif
int main() {#ifdef DYNAMIC
if (enabled) {#endif
printf(“Hello World”);#ifdef DYNAIC
}#endif}
int main() {#ifdef WORLD
printf(“Hello World”);#endif}
AnnotationsUndisciplined
int a = #ifdef DYNAMIC
enabled? 3 : 4 +#endif
5;Align with code structure
Align with variability model
static int _rep_queue_filedone(...)DB_ENV *dbenv;REP *rep;__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUECOMPQUIET(rep, NULL);COMPQUIET(rfp, NULL);return
(__db_no_queue_am(dbenv));#else
db_pgno_t first, last;u_int32_t flags;int empty, ret, t_ret;
#ifdef DIAGNOSTICDB_MSGBUF mb;
#endif// over 100 lines of add. code
}#endif
Variability-AwareParserType SystemStatic AnalysisBug FindingTestingModel CheckingTheorem Proving…
Conflicts
References
Pre
sence
Conditio
ns
true
true
WORLD
BYE
Reachability: pc(caller) -> pc(target)Conflicts: ¬(pc(def1) ˄ pc(def2))
true -> true
true -> (WORLD v BYE)
¬ (WORLD ˄ BYE)
Variability Model:
VM -> (true -> true)
VM -> (true -> (WORLD v BYE))
P
WORLD BYE
VM ->¬ (WORLD ˄ BYE)
40BYEWORLD
AST with Variability Information
Extended Lookup Mechanism
greet.c
printf
msg
VWORLD VBYEprintf
msg msg
main
ε ε
…
true -> true
true -> (WORLD v BYE)¬ (WORLD ˄ BYE)
Formalization: CFJ[ASE’08, TOSEM‘11]
Surface Complexity
Handling Complexity
SATProblem
InherentComplexity
Variability-Aware AnalysisType SystemStatic AnalysisBug FindingTestingModel CheckingTheorem Proving…
CIDE
http://fosd.de/cide
open source, fosd.net/cide
Automated Refactorings• Feature Module • Annotationen
class Stack {int[] data;
#ifdef UNDOint backup;void undo() {
/*...*/ }#endif#ifdef TOP
int top() { /*...*/ }#endif
void push(int o) {#ifdef UNDO
backup = top();#endif
/*...*/}int pop() {
/*...*/ }}
class Stack {int[] data;
#ifdef UNDOint backup;void undo() {
/*...*/ }#endif#ifdef TOP
int top() { /*...*/ }#endif
void push(int o) {#ifdef UNDO
backup = top();#endif
/*...*/}int pop() {
/*...*/ }}
class Stack {int[] data;void push(int o) { …
}int pop() {
/*...*/ }}
class Stack {int[] data;void push(int o) { …
}int pop() {
/*...*/ }}refines class Stack {
int top() { /*...*/ }}
refines class Stack {int top() {
/*...*/ }}refines class Stack {
int backup;void undo() {
/*...*/ }void push(int o) {
backup = top();original(v);
}}
refines class Stack {int backup;void undo() {
/*...*/ }void push(int o) {
backup = top();original(v);
}}
Base
Top
Undo
Automated Transformation
[GPCE’09]
THE REAL-WORLD CHALLENGE
Parse and Type check all configurations of the
entire Linux kernel
10,000 features, 6 million lines of C code
TypeChef
AST with Variability Information
Variability-AwareParsing
greet.c
printf
msg
VWORLD VBYEprintf
msg msg
main
ε ε
…
[OOPSLA‘11]
greet.c
printf
msg
VWORLD VBYE+ printf
+ msg + msg
+ main
ε ε
???
Macro expansion needed for parsing Alternative macros
Undisciplined annotations
Parsing Cwithout Preprocessing
Previous Solutions
Disciplined SubsetRequires Code Preparation
Heuristics and Partial AnalysisInaccurate, False Positives
Brute ForceInfeasible Effort
TypeChef
+
* VA
2 43 5
Variability-Aware
Parser
Variability-Aware
Analysis
https://github.com/ckaestne/TypeChef
Variability-Aware
Lexer
(2
*3
)+
4A5¬A
3 +
4A 4¬A˄B +¬A˄B 6¬A
(
(¬A )¬A
4A (¬A
4¬A˄B +¬A˄B 6¬A
4¬A˄B +¬A˄B 6¬A
)¬A
+
64
VB
6
)
true
4
VA
+
3
Variability-Aware Parsers
GNU C + cpp Java + Antenna
X862.6.33.3
Parsing Linux
Variability in Build System (kbuild) and with #ifdef
Variability Model (kconfig)
0 included header files per C file
0 macros per C file
0 conditional macros per C file
0 alternative macros per C file
0 token per C file
0 % conditional
35385901387
340
33549072
X862.6.33.3
0 C files (x86)
0 seconds per file (median)
0 hours total
0 % overhead (undiscipl.)
0
0 syntax errors
76653085
4.1
0X862.6.33.3
X862.6.33.3
Type Checking Linux
Work in Progress
SUMMARY AND PERSPECTIVE
Summary• Problems:
• Separation of Concerns• Obfuscation• Error-proneness
• Retained advantages:• Easy to use• Flexible• Language-independent• Low adoption barrier
• Improvements:• Views• Visual representation• Variability-aware analysis
• Remaining Problems:• Modular Type Checking• Separate Compilation• Black-box Reuse
“Virtual Separation of Concerns”
Perspective
• Preprocessors common in practice, despite criticism • Neglected by researchers• Tool support can address most problems• Currently scaling to Linux
• Interim solution or serious alternative?
Give preprocessors a second chance!
Questions?
http://fosd.de/cide
Picture Credits
© Stuck in Customs (cc by-nc-sa 2.0)
© Horia Varlan (cc by 2.0)