software runs the world · desktops, laptops, routers, smartphones, tablets! coffee makers, tvs,...

Software runs the world

2

Monday, October 15, 12


• We (sometimes indirectly) interact with devices running (lots of) software every day

2



• We (sometimes indirectly) interact with devices running (lots of) software every day! Desktops, laptops, routers, smartphones, tablets

2




! Coffee makers, TVs, energy meters, medical devices

2




! Coffee makers, TVs, energy meters, medical devices

! Cars, aircraft, weapon systems, nuclear centrifuges

2


Software failures are disruptive

3



• 3/11: Mizuho FG’s ATM system goes down

! 5,600 machines offline for 24 hours

3





• 8/10: Toyota Prius brakes fail due to software glitch

! Ford also issues patch for similar problem

3







• 6/10: Stuxnet malware

! Exploits flaws in industrial control systems

3









• 3/08: Heartland exposes 134M credit cards

! SQL injection used to install spyware

3











• 8/07: LAX offline due to faulty network card

! 17,000 planes grounded for eight hours

3











• 8/07: LAX offline due to faulty network card

! 17,000 planes grounded for eight hours

• 8/03: Northeast, multi-state blackout

! Race condition in power plant management software cascades

3


Software updates are crucial

• Updates: fix bugs, modify behavior, add features

• 2010 NASDAQ hacking incident! investigators blame vulnerability on

out-of-date software

4

Oracle Critical Patch Update Advisory - April 2009DescriptionA Critical Patch Update is a collection of patches for multiple security vulnerabilities. ...

Due to the threat posed by a successful attack, Oracle strongly recommends that customers apply fixes as soon as possible. This Critical Patch Update contains 43 new security fixes across all products.


• Typically require restarting the program

• interrupts active users / processing

• makes services unavailable

5

Software updates are disruptive


Dynamic Software Updating (DSU)

6



• Goal: Update programs while they run ! Avoid interruptions

- Overwhelming number of security breaches due to unpatched software

! Preserve critical program state

6



• Goal: Update programs while they run ! Avoid interruptions

- Overwhelming number of security breaches due to unpatched software

! Preserve critical program state

• Useful for:! Non-stop services

- E.g., Financial processing, air traffic control, network infrastructure

! Programs with long-lived connections

- E.g., OpenSSH and media streaming

! Long-running programs with large in-memory state

- E.g., operating systems, caching servers, in-memory databases

6



• Run program at the old version

• At some point update to the new version, preserving and updating existing program state

• existing connections, important data on the stack and heap, program counter, ...

5Thursday, March 29, 2012





v0 process

Update






v0 process

v0 state Update






v1 code

v0 process

v0 state Update






v1 code

v0 process

v0 state transformed stateUpdate






v1 code

v0 process

v0 state transformed state


upd. process





v1 code

v0 process

v0 state transformed state


Research DSU systems

• C/C++ application support

! PODUS, Ginseng, UpStare, POLUS, Ekiden, Gupta et al

• C/C++ operating systems

! K42, LUCOS, DynaMOS

• Java applications

! Jvolve, JDRUMS, Prose

• Other languages

! *Mod, DLpop, Dynamic ML

6


Many forms of DSU now mainstream

7



language run-times

7



language run-times app. tools

7



language run-times app. tools OSes

7

Bought byOracle in

2011


Practical DSU system features

• Ease of Use

! Update behavior should be easy to reason about

! Should be easy to retrofit existing programs

! Minimize per-update programmer work

• Flexibility

! Support natural program evolution, on-the-fly

• Efficiency

! Quick update times

! Little or no overhead on normal execution

• Scalability

! Support large, real-world programs8


Kitsune, a practical DSU system

9



• Updates the entire program at once

! A dynamic update “restarts” the program with the new code and the existing state

! Only requires simple source-to-source translator

- Employs entirely standard compilation and tools

9







• Update orchestration controlled by developer

! Extra manual effort is worth it: easier to reason about

9







• Update orchestration controlled by developer

! Extra manual effort is worth it: easier to reason about

• New tool called xfgen generates code to transform the existing state

! based on simple developer-written specifications

9


Kitsune: Results

• Applied Kitsune to five open-source programs

! memcached, redis, icecast: 3-6 mos. of releases

! vsftpd, Tor: 2-4 years of releases

• Performance overhead in the noise

• Update times typically less than 40ms

• Programmer effort small

! < 100 LOC per program (largely one-time effort)

! ~ 100 LOC of xfgen specs across all releases

! All comparable to or less than prior systems

10

Flexible

Efficient

Easy to use

Scalable


Kitsune build process

11

Summary:•For each source file

•replace gcc -c with composition of kitc and gcc•Add -shared flag to linker and include kit-rt.a

.c.c

.ckitc gcc -c

-fPIC

-fvis...=

gcc

-shared

.c.c

.c

.c.c

.o

.so

kit-rt.a


Kitsune updating model

12

driver



12

driver !!!!!!!!"#$%&'

()*!+

1.Load first version



12

driver !!!!!!!!"#$%&'

()*!+

1.Load first version2.Run it



12

driver !!!!!!!!"#$%&'

()*!+

,-#-)

1.Load first version2.Run it



12

driver !!!!!!!!"#$%&'

()*!+

,-#-)

1.Load first version2.Run it3.Call back to driver when update ready



12

driver !!!!!!!!"#$%&'

()*!+

,-#-)

!!!!!!"#$%&'

()*!.

1.Load first version2.Run it3.Call back to driver when update ready4.Load second version



12

driver !!!!!!!!"#$%&'

()*!+

!!!!!!"#$%&'

()*!.

,-#-)

1.Load first version2.Run it3.Call back to driver when update ready4.Load second version5.Migrate and transform state



12

driver !!!!!!"#$%&'

()*!.

,-#-)

1.Load first version2.Run it3.Call back to driver when update ready4.Load second version5.Migrate and transform state6.Free up old resources



12

driver !!!!!!"#$%&'

()*!.

,-#-)

1.Load first version2.Run it3.Call back to driver when update ready4.Load second version5.Migrate and transform state6.Free up old resources7.Continue with new version


Programmer obligations

13



• Kitsune DSU requires the programmer to

! Identify where updates may take place

13





! Identify the state to be transferred, and where it should be received in the new code

13





! Identify the state to be transferred, and where it should be received in the new code

! Move the program counter to the right event loop when the new version restarts

13


Example single-threaded server

14

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

!!!A!!!

A

$%-!"#$%&'!:

!!!$%-!8921B!789213

!!!"#00$%5!C!"#8867&>>>'3

!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!!!!!!!78921!C!5)-976%%&8921'3

!!!!!!!78$)%-98660&78921'3

!!!A!!

A

before modification



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A

modification for Kitsune



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3

!!!D$-,@%)9@01#-)&E78$)%-E'3



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3

!!!D$-,@%)9@01#-)&E78$)%-E'3

!!!D$-,@%)9169#@-6"$5*#-)&'3

==!#@-6"$5*#-)1



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3

!!!D$-,@%)9@01#-)&E78$)%-E'3

!!!$2!&FD$-,@%)9$,9@01#G%5&''!:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A

!!!D$-,@%)9169#@-6"$5*#-)&'3

==!#@-6"$5*#-)1



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3

!!!D$-,@%)9@01#-)&E78$)%-E'3

!!!$2!&FD$-,@%)9$,9@01#G%5&''!:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A

!!!D$-,@%)9169#@-6"$5*#-)&'3

HIJK9LIMNLO

==!#@-6"$5*#-)1



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3

!!!D$-,@%)9@01#-)&E78$)%-E'3

!!!$2!&FD$-,@%)9$,9@01#G%5&''!:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A

!!!D$-,@%)9169#@-6"$5*#-)&'3

HIJK9LIMNLO

==!#@-6"$5*#-)1

!!!$2!&FPQRSNJK9LIMNL&8921''!:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3

!!!D$-,@%)9@01#-)&E78$)%-E'3

!!!$2!&FD$-,@%)9$,9@01#G%5&''!:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A

!!!D$-,@%)9169#@-6"$5*#-)&'3

!!!$2!&FD$-,@%)9$,9@01#G%592*6"

!!!!!!!!!!&T78$)%-U''!:

!!!!!!PQRSNJK9LIMNL&78921'3

!!!A

HIJK9LIMNLO

==!#@-6"$5*#-)1


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A



15

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53

(6$1!78$)%-98660&$%-!21'!:

!!!!;<$8)!&+'!:

!!!!!!!==!>>>!0*67),,!78$)%-!*)?@),-,

A!!!A

$%-!"#$%!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&'!:

!!!$%-!8921B!789213

!!!!!"#00$%5!C!"#8867&>>>'3

!!!!!8921!C!,)-@0976%%&'3

!!!;<$8)!&+'!:

!

!!!!!!78921!C!5)-976%%&8921'3

!!!!!!78$)%-98660&78921'3

!!!A

A


!!!D$-,@%)9@01#-)&E"#$%E'3

!!!D$-,@%)9@01#-)&E78$)%-E'3

!!!$2!&FD$-,@%)9$,9@01#G%5&''!:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A

!!!D$-,@%)9169#@-6"$5*#-)&'3

!!!$2!&FD$-,@%)9$,9@01#G%592*6"

!!!!!!!!!!&T78$)%-U''!:

!!!!!!PQRSNJK9LIMNL&78921'3

!!!A

HIJK9LIMNLO

==!#@-6"$5*#-)1


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!A

!!!!!!78$)%-98660&78921'3


Multithreading support

• All threads synchronize at update points

! Assumes threads will reach update points eventually

! Assumes they hold no resources (e.g., locks)

• Main thread restarts (as in the ST case)

• Restarts child threads at update points

! Hijack threads library to track pthread function, args

• Done when all threads reach their update points after restarting

16


Migrating and transforming state

• State may need to be transformed to work with the new program

! Transformation piggybacks on top of migration

17





17

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53old





17

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53old

-/0)1)2!7<#*!41#-#3

1#-#!4"#00$%53new





17

!"#$%&'($)&*+%$!$",$-./%$"#$#$01$-(%$#+11012$/#"2#&3

&14$0-5$'"##%5/"14012$*"'&6"1$%$01$-(%$1%7$/#"2#&3$

4"

!!!&%'(')#**+,-./0

'''12%3425-&%6.678"76!/0

%14

Migration

Xform

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53old

-/0)1)2!7<#*!41#-#3

1#-#!4"#00$%53new





17

!"#$%&'($)&*+%$!$",$-./%$"#$#$01$-(%$#+11012$/#"2#&3

&14$0-5$'"##%5/"14012$*"'&6"1$%$01$-(%$1%7$/#"2#&3$

4"

!!!&%'(')#**+,-./0

'''12%3425-&%6.678"76!/0

%14

Migration

Xform

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53old

-/0)1)2!7<#*!41#-#3

1#-#!4"#00$%53new

1%7883&/59$:$"*4883&/59;

1%7883&//012$:$3&**"'<1%7883&/59=509%",<'(&#=>>;

,"#$<01-$0:?;0@1%7883&/59;0AA>$B

$$"*4884&-&$C$:$"*4883&//012D0E;

$$1%7884&-&$=/$:$F1%7883&//012D0E;

!!40!C!"#8867&H'3

!!,%0*$%V&40BHBUW1UBX'3

G





17

-/0)1)2!$%-!1#-#3

1#-#!4"#00$%53old

-/0)1)2!7<#*!41#-#3

1#-#!4"#00$%53new

Xfgen tool•Automate migration code•Require programmer to write relevant xform code using high-level specs


xfgen specs

18


xfgen specs

• INIT tgt: { action }

! Initializes new variable or values of a new type

- tgt is a global/local variable, or a C type or a struct field

- action is C-like code for initializing it

18


xfgen specs





• tgt " tgt: { action }

! Copies and transforms changed variables/values

- action may refer to old value when initializing the new one

18


xfgen specs





• tgt " tgt: { action }

! Copies and transforms changed variables/values

- action may refer to old value when initializing the new one

• xfgen generates C code from these specs and integrates it with generated migration code

18


Example 1

19

$%-!60976@%-3old$%-!,)-976@%-3

$%-!5)-976@%-new


Example 1

19

$%-!60976@%-3old$%-!,)-976@%-3

$%-!5)-976@%-new

QHQJ!5)-976@%-Y!:!Z6@-!C![66*&Z681,/"&60976@%-'!=!.'3!A

QHQJ!,)-976@%-Y!:!Z6@-!C!7)$8$%5&Z681,/"&60976@%-'!=!.'3!A

xfgen spec


Example 1

19

$%-!60976@%-3old$%-!,)-976@%-3

$%-!5)-976@%-new

QHQJ!5)-976@%-Y!:!Z6@-!C![66*&Z681,/"&60976@%-'!=!.'3!A

QHQJ!,)-976@%-Y!:!Z6@-!C!7)$8$%5&Z681,/"&60976@%-'!=!.'3!A

xfgen spec

9+4"!9D$-,@%)9-*#%,26*"95)-976@%-&'!:

!!42$!4681960976@%-!C!&42$4'!D$-,@%)9866D@09D)/9681&T60976@%-U'3

!!42$!4%);95)-976@%-!C!&42$4'!D$-,@%)9866D@09D)/9%);&T5)-976@%-U'3

!!4%);95)-976@%-!C![66*&4681960976@%-!=!.'3

A

9+4"!9D$-,@%)9-*#%,26*"9,)-976@%-&'!:!!H=$&5$&I")%$=H!A

generated code


Example 2

20

$:%;";<!42$!1#-#3

1$3=,$!8$,-!:

!!42$!D)/3

!!1#-#!4(#83

!!1$3=,$!8$,-!4%)X-3

A!4"#00$%53

old new

$:%;";<!*+2>!1#-#3

1$3=,$!8$,-!:

!!42$!D)/3

!!1#-#!4(#83

!!42$!7$13

!!1$3=,$!8$,-!40%)X-3

A!4"#00$%53


Example 2

20

$:%;";<!1#-#!\]!$:%;";<!1#-#Y!:!Z6@-!C!&*+2>'Z$%3!A

QHQJ!1$3=,$!8$,->7$1!:!Z6@-!C!\+3!A

,-*@7-!8$,->%)X-!\]!,-*@7-!8$,->0%)X-

xfgen spec

$:%;";<!42$!1#-#3

1$3=,$!8$,-!:

!!42$!D)/3

!!1#-#!4(#83

!!1$3=,$!8$,-!4%)X-3

A!4"#00$%53

old new

$:%;";<!*+2>!1#-#3

1$3=,$!8$,-!:

!!42$!D)/3

!!1#-#!4(#83

!!42$!7$13

!!1$3=,$!8$,-!40%)X-3

A!4"#00$%53


Example 3

21

1#-#!44"#00$%53old ,-*@7-!8$,-!:

!!$%-!D)/3

!!1#-#!4(#83

!!,-*@7-!8$,-!4%)X-3

A!4"#00$%53

new


Example 3

21

1#-#!44"#00$%53old ,-*@7-!8$,-!:

!!$%-!D)/3

!!1#-#!4(#83

!!,-*@7-!8$,-!4%)X-3

A!4"#00$%53

new

"#00$%5!\]!"#00$%5!:

!!42$!D)/3

!!Z6@-!C!H^LL3

!!<+3&D)/!C!_3!D)/!`!Z681,/"&76%a59,$b)'3!D)/cc'!:

!!!!4<!&Z$%dD)/e!FC!_'!:

!!!!!!Z%);-/0)&1$3=,$!8$,-'!47@*!C

!!!!!!!!"#8867&14?;+<&Z%);-/0)&1$3=,$!8$,-'''3

!!!!!!7@*\]D)/!C!D)/3

!!!!!!7@*\](#8!C!Z$%dD)/e3

!!!!!!7@*\]%)X-!C!Z6@-3

A!!A!A xfgen spec


xfgen C extensions

• Convenient syntax to hide complexity of dealing with two versions:

! $out, $in - address of input/output of rule

! $newsym(name), $oldsym(name) - return address of new/old version variable or function

! $newtype(type name), $oldtype(type name) - refer to the new or old types from xfgen code

! $xform(old_type, new_type) - returns the address of a function to transform between two types

22


vsftpd transformation example

standalone.c/s_p_ip_count_hash -> standalone.c/s_p_ip_count_hash: {

$oldtype(struct hash) *in_hash = ($oldtype(struct hash) *)$in;

int i;

$out = ($newtype(struct hash) *)hash_alloc(256, sizeof($newtype(struct in6_addr)),

sizeof(unsigned int), $newsym(hash_ip));

/* for each bucket */

for (i=0; i<in_hash->buckets; i++) {

$oldtype(struct hash_node) *old_p_node = in_hash->p_nodes[i];

/* for each bucket element, rehash */

while(old_p_node != NULL) {

void *ip = xform_ip_address(old_p_node->p_key);

hash_add_entry(($newtype(struct hash) *)$out, ip, old_p_node->p_value);

free(ip);

old_p_node = old_p_node->p_next;

}

}

}

23


Using Kitsune and xfgen

24

.c.c

.ckitc gcc -c

-fPIC

-fvis...=

gcc

-sharedxfgen

.c.c

.ts

.xf

.c.c

.c

.c.c

.o

.so

st.c rt.a

.c.c

.ts

(old)


Using Kitsune and xfgen

24

• Transformation specs in per-update .xf file• Linked in with new version and invoked by D$-,@%)9169#@-6"$5*#-)&' and PQRSNJK9LIMNL&'

.c.c

.ckitc gcc -c

-fPIC

-fvis...=

gcc

-sharedxfgen

.c.c

.ts

.xf

.c.c

.c

.c.c

.o

.so

st.c rt.a

.c.c

.ts

(old)


Annotations

25


Annotations

• To generate traversal code, we look at the types

25


Annotations


• Add annotations to help code generator

! KS_PTRARRAY(S) – size of pointed-to array

25

1$3=,$!f@g)*!:

!!42$!%3

!!1#-#!4JKLMNOPOOPQ<1>!f3

A3


Annotations




! KS_ARRAY(S) – size of array

25

1$3=,$!$%8$%)f@2!:

!!42$!%3

!!1#-#!JKLPOOPQ<1>!fde3

A3


Annotations





! KS_OPAQUE – non-traversed pointer

25


Annotations





! KS_OPAQUE – non-traversed pointer

! KS_FORALL(@t) – polymorphism intro.

! KS_VAR(@t) – refer to type var

! KS_INST(typ) – instantiate poly. type

25


Polymorphic types

26

$;)%*#$;@AB'

1$3=,$'8$,-!:

!!!J!4(#83!

!!!8$,-`J]!4%)X-3

A! C++

1$3=,$'8$,-!:

!!!9+4"!4(#83!

!!!1$3=,$!8$,-!4%)X-3

A3C


Polymorphic types

26

$;)%*#$;@AB'

1$3=,$'8$,-!:

!!!J!4(#83!

!!!8$,-`J]!4%)X-3

A! C++

1$3=,$!8$,-!:

!!9+4"!JKLRPO<S->!4(#83

!!1$3=,$!8$,-!JKLTUKN<S->!4%)X-3

A!JKL!VOPWW<S->3

annotated C

1$3=,$'8$,-!:

!!!9+4"!4(#83!

!!!1$3=,$!8$,-!4%)X-3

A3C


Polymorphic types

26

$;)%*#$;@AB'

1$3=,$'8$,-!:

!!!J!4(#83!

!!!8$,-`J]!4%)X-3

A! C++

1$3=,$!8$,-!:

!!9+4"!JKLRPO<S->!4(#83

!!1$3=,$!8$,-!JKLTUKN<S->!4%)X-3

A!JKL!VOPWW<S->3

annotated C

• Given such a definition, xfgen can generate transformers for instances of type ,-*@7-!8$,-

! Given ,-*@7-!8$,-!hO9QHOJ&1#-#'!48, can apply transformation to generic type:

! 1#-#!\]!1#-#!:!Z6@-!C!>>>!Z$%!>>>!A

1$3=,$'8$,-!:

!!!9+4"!4(#83!

!!!1$3=,$!8$,-!4%)X-3

A3C


Benchmark programs

• Very secure FTP daemon - file transfers, securely

• Redis - key/value server

• Tor - anonymous routing daemon

• Memcached - caching daemon

• Icecast - streaming music server

27


!"#$F"&'&"(

)*+,-.*C,0##"$*$1"#

2.3*$,4"1#$-

51$-6#,!*&&-

7B9,#:F*#-B"F+-

!"#"$%&'()ABCD)C#E'F0%#1CB

;*<"F1$=>"B>51$-6#,>C?*#9,->*F,>1#>$?,>1#1$1*&>@,F-1"#

!?*#9,->B"F>A#"F$>B>C#$F6-1"#>D,$,C$1"#>A=-$,+

Steady state performance overhead

28

• Overall: -2.18% to 1.79% overhead (in the noise)

• Also: UpStare (different platform): 4.9% and 7.4% for vsftpd


Update times

29

• < 40ms in all cases but icecast


Update times, by state size

34

• Key difference is data representation: arrays vs. nested objects with pointers to static memory

!"

!#"

!$""

!$#"

!%""

!" !%""" !&""" !'""" !(""" !$"""" !$%""" !$&"""

)*+,-.!-/0.!1023

4!5.678,9).!*,/:2

:.+/2!8"7;8$

:.+/2!8$7;8%

:.+/2!8%7;8<

:.+/2!8<7;8&

:.+/270=+!8"7;8$

:.+/270=+!8$7;8%

:.+/270=+!8%7;8<

:.+/270=+!8<7;8&

0.0>,>?.+!8"7;8$

0.0>,>?.+!8$7;8%

Monday, October 15, 12 30

Conclusions

• Kitsune treats DSU as a program feature! Deliberately makes update semantics apparent to

programmer

! But, programmers only need to “pay for what they use”

- Effort for changing program roughly corresponds to effort in thinking about DSU

• Gives C programmers the control they need

• Results promising! Applied to 35 updates of 5 real applications

! Most easy-to-use, efficient, and flexible C DSU system to date

39


Questions?


software runs the world · desktops, laptops, routers, smartphones, tablets! coffee makers, tvs,...

Documents