isca needle a_0610
DESCRIPTION
Moving the Needle: Computer Architecture Research in Academe and IndustryBy Bill DallyTRANSCRIPT
![Page 1: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/1.jpg)
Moving the NeedleComputer Architecture Research
in Academe and IndustryBill DallyChief Scientist & Sr. VP of Research, NVIDIABell Professor of Engineering, Stanford University
![Page 2: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/2.jpg)
Outline
The Research FunnelMost ideas fail
Those that succeed take 5-10 years
The Research Formula
Constraints
The Academic Advantage
The Industrial Advantage
Startups
Best practices
![Page 3: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/3.jpg)
Goal – Positive Impact on a Product
![Page 4: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/4.jpg)
The Research Funnel
Applications
Technology
ConceptDev
Model Eval Dev
insight
![Page 5: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/5.jpg)
Most ideas fail
The ideas that succeed take a long time
ConceptDev
Model Eval Dev
![Page 6: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/6.jpg)
Most ideas fail
The ideas that succeed take a long time
ConceptDev
Model Eval Dev
![Page 7: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/7.jpg)
Most ideas fail
So terminate the bad ones quickly
![Page 8: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/8.jpg)
Most ideas fail
So terminate the bad ones quickly
Be a terminator, not an advocate
![Page 9: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/9.jpg)
Dally, “Micro-Optimization of Floating-Point Operations, ASPLOS, 1989, pp 283-289
![Page 10: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/10.jpg)
![Page 11: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/11.jpg)
Most ideas fail
The ideas that succeed take a long time
ConceptDev
Model Eval Dev
![Page 12: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/12.jpg)
The ideas that succeed take a long time
So aim research 5-10 years ahead of current practice
![Page 13: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/13.jpg)
Current Architecture Practice
![Page 14: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/14.jpg)
![Page 15: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/15.jpg)
5-10 years
Aim Here
![Page 16: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/16.jpg)
5-10 years
Enable this point
![Page 17: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/17.jpg)
Timeline for some ideas
Idea Concept Published Product DT
Stream Processing 1995 1998 2006 11
Virtual Channels 1985 1990 1992 7
Equalized Signaling 1995 1996 2000 5
High-Radix Networks 2002 2005 2008 6
![Page 18: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/18.jpg)
The Performance Equation
ckf
CPINITime
![Page 19: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/19.jpg)
The Research Formula
ROI reward
risk effort
![Page 20: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/20.jpg)
Reward
If you are wildly successful, what difference will it make?
ROI reward
risk effort
![Page 21: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/21.jpg)
Effort
Learn as much as possible with as little work as possible
ROI reward
risk effort
![Page 22: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/22.jpg)
Effort
Do the minimum analysis and experimentation necessary to make a point
ROI reward
risk effort
![Page 23: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/23.jpg)
Real and Artificial Constraints
Real Constraints Artificial Constraints
Laws of physicsFuture semiconductor processesPackaging and thermal limitsFuture applications
Existing ISAExisting OSToday’s benchmarksExisting compilersInfrastructure
![Page 24: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/24.jpg)
Constraining Infrastructure
uArch Idea
Other
uArch
ISA
Compiler
Benchmarks
Binaries
Simulator
![Page 25: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/25.jpg)
Constraining Infrastructure
uArch Idea
Other
uArch
ISA
Compiler
Benchmarks
Binaries
Simulator
![Page 26: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/26.jpg)
Constraining Infrastructure
uArch Idea
Other
uArch
ISA
Compiler
Benchmarks
Binaries
Simulator
![Page 27: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/27.jpg)
The contribution is insight
Not novelty
Not numbers
![Page 28: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/28.jpg)
Research is a hunt for insight
Need to get off the beaten path to find new insights
![Page 29: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/29.jpg)
Road-Kill Research
uArch Idea
Other
uArch
ISA
Compiler
Benchmarks
Binaries
Simulator
![Page 30: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/30.jpg)
![Page 31: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/31.jpg)
Looking here for lost keys
![Page 32: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/32.jpg)
Lost keys here
Looking here
![Page 33: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/33.jpg)
The Academic Advantage
![Page 34: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/34.jpg)
The Academic Advantage
Freedom
![Page 35: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/35.jpg)
The Academic Advantage
Freedom from artificial constraints
Freedom to fail (take risks)
![Page 36: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/36.jpg)
Academic research matched for early stages of the funnel
ConceptDev
Model Eval Dev
![Page 37: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/37.jpg)
Example: ELM
An Ensemble Many Ensembles and memory tiles on a die
37
![Page 38: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/38.jpg)
Example: ELM
Balfour et al., "An Energy-Efficient Processor Architecture for Embedded Systems" CAL, Jan. 2008, pp 29-32.
![Page 39: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/39.jpg)
ELM Infrastructure
uArch Idea
Other
uArch
ISA
Compiler
Benchmarks
Binaries
Simulator
Changed for ELM
![Page 40: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/40.jpg)
The Industrial Advantage
Resources and Experience
![Page 41: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/41.jpg)
The Industrial Advantage
Resources to carry out detailed studies
Experience to address commercial constraints
![Page 42: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/42.jpg)
The ideal partnership:
Academic research 5-10 years out, focused on industry problems
Transfer insight to industrial research to refine into product
ConceptDev
Model Eval Dev
![Page 43: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/43.jpg)
What transfers is insight
Not academic design
Not performance numbers
![Page 44: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/44.jpg)
What transfers is insight
And its transferred by people
Not papers
![Page 45: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/45.jpg)
Concept
Analysis
Simulation
Prototype
Refine Concept
Detailed Design
Academic
Industrial
![Page 46: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/46.jpg)
Concept
Analysis
Simulation
Prototype
Refine Concept
Detailed Design
Academic Industrial
Gap
Paper Impact
![Page 47: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/47.jpg)
Example: Cray T3D and T3E
![Page 48: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/48.jpg)
J-Machine
• MIT 1987-1992
• 3-D network
• Global address space
• Fast messaging and synchronization
• Support for many models of computation
![Page 49: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/49.jpg)
Cray T3D• Started working with Cray in
1989
• Project started early 1990
• First ship in mid 1992
• From J-Machine• Network
• Fast communication/sync
• Global address space
• For reality• Alpha processors
• MECL gate arrays
• Robust software stack
![Page 50: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/50.jpg)
Best Practices for Academics
• Long-term perspective (5-10 years)• Know your customer and their long-term issues
• Look at tomorrow’s applications, not yesterdays
• Maximize reward, minimize effort• Estimate maximum impact – terminate…
• Minimal analysis and experiment to make the point
• Exploit your freedom• Don’t be limited by exiting tools, benchmarks, ISAs, …
• Carry result to impact
• Build relationships with industry
ROI reward
risk effort
uArch Idea
Other
uArch
ISA
Compiler
Benchmarks
Binaries
Simulator
![Page 51: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/51.jpg)
Best Practices for Industry
• Leverage academic research• Build partnerships
• Articulate long-term research issues
• Be open-minded
• Minimize artificial constraints
• Carry concepts across “the gap”
• Open infrastructure
![Page 52: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/52.jpg)
A Partnership
Academe Industry
Filtered, De-risked Concepts
Future issuesInfrastructure
![Page 53: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/53.jpg)
The Startup Path
When you can’t find an appropriate industrial partner, make one.
STAC, Avici, Velio, SPI
![Page 54: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/54.jpg)
Concept
Analysis
Simulation
Prototype
Refine Concept
Detailed Design
Academic
Startup
![Page 55: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/55.jpg)
Startup Pros/Cons
Pros
• Don’t have to convince existing company to change course (until exit)
Cons
• Have to convince investors (repeatedly)
• Have to build a whole company, not just a development team• Finance, sales, marketing, …
• Limited resources
• Impatient capital
![Page 56: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/56.jpg)
Example: SPI
Date Event
Jan 2004 SPI Incorporated
Nov 2004 First round financing
April 2006 Tapeout Storm-1
Oct 2006 First ship of Storm-1
2007 Software, software, software
2008 Customers in production
Sept 2009 Doors close
![Page 57: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/57.jpg)
Much easier to license technology to an existing company
![Page 58: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/58.jpg)
Starting a company to bring a new semiconductor product to market costs $30M (to cash flow positive)
If it’s a programmable processor, its $70M
Investors want a 10x ROI
Need to see a $700M exit to justify a new processor company
![Page 59: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/59.jpg)
The future of computer architecture
![Page 60: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/60.jpg)
The future of computer architecture
• NOW is an ideal time for research to move the needle
• Computers are drastically changing• Pervasive parallelism
• Energy limited
• Bandwidth constrained
• Opportunity to set the MSB of future computers in the next few years
• Requires changing the whole stack
• Requires industry-academe partnership
![Page 61: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/61.jpg)
Energy-Efficient ArchitectureAbstracting Locality
20mm
7pJ
50pJ 500pJ
2000pJ
2000pJ
P P P P
L1 L1 L1 L1
Net
L2
Net
L3
![Page 62: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/62.jpg)
Solution involves many levels of the “stack”
Application
Algorithm
Prog. System
Compiler
ISA
uArch
Design
Circuits
Process
Too constrained to innovate within one layer
![Page 63: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/63.jpg)
Industry
Academe
ROI reward
risk effort
uArch Idea
Other
uArch
ISA
Compiler
Benchmarks
Binaries
Simulator
![Page 64: Isca needle a_0610](https://reader034.vdocuments.net/reader034/viewer/2022042606/54b7aa984a79594b258b4652/html5/thumbnails/64.jpg)
Moving the NeedleComputer Architecture Research
in Academe and IndustryBill DallyChief Scientist & Sr. VP of Research, NVIDIABell Professor of Engineering, Stanford University