18 meta techniques in computer science
TRANSCRIPT
Kanazawa Institute of Technology
18 Meta Techniques in
Computer Science
May 1st, 2015
Jun Nakano Kanazawa Institute of Technology, Japan
http://www.nakanolab.net/
K.I.T. 18 Meta Techniques in Computer Science 1. Caching 2. Blocking vs. Non-blocking 3. Profiling 4. Pipelining 5. Speculation (Prediction) 6. Relaxation 7. Eager vs. Lazy 8. Filtering 9. Reuse
10. Parallelization 11. Coupling vs. Decoupling 12. Multilevel / Clustering 13. Lookahead / Prefetching 14. Dynamic (Adaptive) 15. Replication 16. Virtualization 17. Checkpoints / Snapshots 18. Transactions
I have been seeing some common techniques used across various disciplines of computer science. Above is the list of 18 such "meta techniques" in no particular order. In the following slides, I will describe ideas behind them and show some examples (somewhat biased toward computer architecture).
2 18 Meta Techniques in Computer Science
K.I.T. #1 Caching #2 Blocking vs. Non-blocking #3 Profiling #4 Pipelining #5 Speculation (Prediction) #6 Relaxation #7 Eager vs. Lazy #8 Filtering #9 Reuse #10 Parallelization #11 Coupling vs. Decoupling #12 Multilevel / Clustering #13 Lookahead / Prefetching #14 Dynamic (Adaptive) #15 Replication #16 Virtualization #17 Checkpoints / Snapshots #18 Transactions
3 18 Meta Techniques in Computer Science
K.I.T. #1 Caching Idea
Exploit » Spatial locality » Temporal locality » 80-20 rule
Examples
Data cache Instruction cache: Static & Dynamic (trace cache of Pentium 4) Buffer cache of file system and DBMS Web browser's cache Proxy server Memoization LZW compression (caching frequent patterns in a dictionary)
4 18 Meta Techniques in Computer Science
K.I.T. #2 Blocking vs. Non-blocking Idea
Do useful stuff while waiting for something to complete
Examples Multi-tasking (OS scheduler) Multi-threading Asynchronous I/O Asynchronous communication in MPI (Message Passing
Interface): Overlapping computation and communication Out-of-order vs. in-order execution in processor Allow cache reads while there are outstanding cache misses
(MSHR: Miss Status Holding Register)
5 18 Meta Techniques in Computer Science
K.I.T. #3 Profiling Idea
Use runtime information to optimize execution
Examples When optimizing code, start with "hot spots" (via gprof, etc.) Lots of work by compiler folks (e.g., code layout optimization
by Spike)
6 18 Meta Techniques in Computer Science
K.I.T. #4 Pipelining Idea
For better throughput
Examples Processor pipeline (fetch, decode, execute, memory, and write
back)
A
B
C
D
A
B
C
D
Task
s
Time Time Wash
Dry
Fold
Sequential Laundry Pipelined Laundry
Source: http://www.cs.berkeley.edu/~pattrsn/252S01/
7 18 Meta Techniques in Computer Science
K.I.T. #5 Speculation (Prediction) Idea
Make an educated guess and believe you are lucky
Examples
Branch prediction Prefetching Value prediction Speculative helper thread to hide cache miss latency of main
thread
8 18 Meta Techniques in Computer Science
K.I.T. #6 Relaxation Idea
What if you remove a constraint?
Examples Relaxed memory consistency in multiprocessor systems
» Sequential consistency: Intuitive for programmers but little room for hardware optimization
» Relaxed consistency: Pursue hardware optimization (e.g., write buffer) at the cost of some inconveniences to programmers
Eventual consistency for distributed hash tables
9 18 Meta Techniques in Computer Science
K.I.T. #1 Caching #2 Blocking vs. Non-blocking #3 Profiling #4 Pipelining #5 Speculation (Prediction) #6 Relaxation #7 Eager vs. Lazy #8 Filtering #9 Reuse #10 Parallelization #11 Coupling vs. Decoupling #12 Multilevel / Clustering #13 Lookahead / Prefetching #14 Dynamic (Adaptive) #15 Replication #16 Virtualization #17 Checkpoints / Snapshots #18 Transactions
10 18 Meta Techniques in Computer Science
K.I.T. #7 Eager vs. Lazy Idea
Eager » Do it now if you are sure you will have to do it later, or if it is safe
and cheap Lazy
» Otherwise, do nothing until you really have to
Examples Demand paging (lazy) Copy on write (lazy) Some data structures such as Fibonacci heap that aim at
reducing amortized cost (lazy) Just-in-time compiler (eager / lazy) Protocols in MPI (Message Passing Interface)
» Eager: Send small data eagerly to avoid negotiation cost » Rendezvous: Negotiate with receiver before sending large data
11 18 Meta Techniques in Computer Science
K.I.T. #8 Filtering Idea
Avoid going down to the lower level if possible
Examples Cache hierarchy
» L1 cache → L2 cache → main memory Bloom filter
12 18 Meta Techniques in Computer Science
K.I.T. #9 Reuse Idea
Do not repeat the same thing » If the computation of f(x) is expensive and occurs frequently,
remember the answer
Examples Memoization Dynamic Instruction Reuse (Sodani and Sohi, ISCA 1997)
13 18 Meta Techniques in Computer Science
K.I.T. #10 Parallelization Idea
Double the computational resources and cut the time in half » In practice, be aware of Amdahl's law
Examples
SMP Multicore Cluster of computers MapReduce
14 18 Meta Techniques in Computer Science
#11 Coupling vs. Decoupling Idea
Less dependency is easier to handle but choose design point wisely considering trade-offs
Examples
Cache » L1 cache: Data / Instruction (Harvard architecture) » L2 cache: Unified
Decoupling is often preferred in software engineering » Interface and Implementation (object-oriented programming) » Policy and Mechanism (computer networks) » Control plane and data plane in SDN (Software Defined
Networking) » Protocol stack of TCP/IP (physical, data link, network, transport,
and application) » Database normalization
15 18 Meta Techniques in Computer Science
K.I.T. #12 Multilevel / Clustering Idea
Organize things neatly
Examples Cache hierarchy Directory structure of file systems Grouping instructions in processor pipeline TCP/IP Anything that looks like a tree
16 18 Meta Techniques in Computer Science
K.I.T. #1 Caching #2 Blocking vs. Non-blocking #3 Profiling #4 Pipelining #5 Speculation (Prediction) #6 Relaxation #7 Eager vs. Lazy #8 Filtering #9 Reuse #10 Parallelization #11 Coupling vs. Decoupling #12 Multilevel / Clustering #13 Lookahead / Prefetching #14 Dynamic (Adaptative) #15 Replication #16 Virtualization #17 Checkpoints / Snapshots #18 Transactions
17 18 Meta Techniques in Computer Science
#13 Lookahead / Prefetching Idea
Right thing at the right time
Examples Hardware/software prefetching from memory Cache line
18 18 Meta Techniques in Computer Science
K.I.T. #14 Dynamic (Adaptive) Idea
Effective use of runtime information while running
Examples Just-in-time compilation of Java Dynamic optimization in LLVM Load balancer for Web server farm
19 18 Meta Techniques in Computer Science
K.I.T. #15 Replication Idea
Better reliability and/or locality
Examples Disk mirroring Replication in distributed databases Replication in distributed hash tables CDN (Content Delivery Network): Akamai, etc. GFS (Google File System)
20 18 Meta Techniques in Computer Science
K.I.T. #16 Virtualization Idea
Beyond physical limitations Redirection
Examples Virtual memory Virtual devices Virtual machines Virtual file systems View in database Shortcuts in Windows and symbolic links in UNIX
Virtual Illusion
Physical Entity
21 18 Meta Techniques in Computer Science
K.I.T. #17 Checkpoints / Snapshots Idea
Avoid losing data in case of failures / human errors » Be aware of output commit problem
Examples
Database Copy on write
22 18 Meta Techniques in Computer Science
K.I.T. #18 Transactions Idea
Impose atomicity for group of operations
Examples Database Transactional memory for parallel programming
» Trade-off between correctness and performance • Possible to write fast and correct programs using locks, etc.,
but error-prone • No worry about locking and unlocking in transactional
memory at the cost of some performance overhead
23 18 Meta Techniques in Computer Science
K.I.T. Summary 1. Caching 2. Blocking vs. Non-blocking 3. Profiling 4. Pipelining 5. Speculation (Prediction) 6. Relaxation 7. Eager vs. Lazy 8. Filtering 9. Reuse
10. Parallelization 11. Coupling vs. Decoupling 12. Multilevel / Clustering 13. Lookahead / Prefetching 14. Dynamic (Adaptive) 15. Replication 16. Virtualization 17. Checkpoints / Snapshots 18. Transactions
Some advice for you if you are to apply these meta techniques to your project: Be aware of trade-offs that exist in almost any engineering project As they say, "Devil is in the details."
It is most appropriate to consider these meta techniques as just a food for thought.
24 18 Meta Techniques in Computer Science