research industry panel review
DESCRIPTION
Summary of ICSE 2011 Panel on "What Industry wants from Research". This is a summary of all the presentations from that panel that I presented in an invited talk at the CSER meeting in Toronto, November, 2011.TRANSCRIPT
![Page 1: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/1.jpg)
Highlights from an ICSE 2011 Panel: What Industry Wants from Research
Panel Organizers: Jorge Aranda, Daniela Damian, Marian Petre, Margaret-‐Anne Storey, Greg Wilson
1
![Page 2: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/2.jpg)
2
![Page 3: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/3.jpg)
A conversaEon is started…
3
![Page 4: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/4.jpg)
QuesEons we asked…
What is your percep0on of soMware engineering research? Conferences, journals, collaboraEons…
What counts as evidence? How do you find out about results?
What topics should soMware engineering research focus on?
4
![Page 5: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/5.jpg)
Percep0ons of so7ware research
Focused on organiza0ons that don’t compare to us
Results may fail to scale
Piecemeal improvements
Irrelevant, dated, ignoring cuWng edge problems
hXp://Enyurl.com/icse2011panel 5
![Page 6: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/6.jpg)
“[I’m afraid] that industrial so9ware engineers will think that I’m now doing academic so9ware engineering and then not listen to me. (…) if I start talking to them and claim that I’m doing so9ware engineering research, a9er they stop laughing, they’re gonna stop listening to me. Because it’s been so long since anything actually relevant to what pracFFoners do has come out of that environment, or at least the percentage of things that are useful that come out of that environment is so small.”
6
![Page 7: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/7.jpg)
What counts as evidence?
Preference for quanEtaEve data and staEsEcal significance
7
![Page 8: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/8.jpg)
What counts as evidence? Preference for quanEtaEve data and staEsEcal
significance
“managers are coin operated in some sense. If you can’t quanFfy it in terms of Fme or in terms of money, it doesn’t make much difference to them. (…) I think there does need to be some noFon of a numeric or at least an objecFve measure.”
8
![Page 9: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/9.jpg)
What counts as evidence? Preference for quanEtaEve data and staEsEcal
significance
AssociaEon of qualitaEve data with 'mere anecdote‘
And yet…. InfluenEal opinions persuasive “I trust the people I hire”
9
![Page 10: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/10.jpg)
Difficulty in applying our findings
Resistance to change for small potenEal gains
“(…) it would depend in part of how cumbersome your techniques are; how much retraining I’m going to have to do on my staff. (…) I might decide that even if you’re legit and you actually do come up with 15%, that that’s not enough to jusFfy it.”
10
![Page 11: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/11.jpg)
Dissemina0on of results
Results are not geWng out there! SuggesFons:
Researchers should be aXending pracEEoner conferences
Need to disEll research results for pracEEoners
11
![Page 12: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/12.jpg)
Research topics of interest
Developer issues Tool issues
Code issues
User Issues
EvaluaEon Issues
Management issues
12
![Page 13: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/13.jpg)
Reac0ons to our blog?
hXp://Enyurl.com/icse2011panel
13
![Page 14: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/14.jpg)
Broadening this conversaEon
14
![Page 15: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/15.jpg)
QuesEons we posed to the panelists…
What percep0on does industry have of academic research?
What kind of evidence is compelling?
Which research ques0ons should be addressed?
Most useful empirical finding? Success story?
How to improve dissemina0on of research results?
How to engage industry in research?
15
![Page 16: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/16.jpg)
Panel parEcipants
David Weiss, Iowa State University John Penix, Google
Lionel Briand, Simula Research Laboratory
Peri Tarr, IBM Thomas J. Watson Research Center
Tatsuhiro Nishioka, Corporate SoMware Engineering Center, Toshiba CorporaEon
Wolfram Schulte, MicrosoM Research
16
![Page 17: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/17.jpg)
Panel Highlights: David Weiss, perspec0ve from Avaya
17
![Page 18: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/18.jpg)
Panel Highlights: John Penix, Google
• Striving for conEnuous improvement of useful tools: – “We can’t improve what we can’t measure” – “Our goal: make the tools disappear from the workflow”
• In terms of scale…. 5000 devs in 40 offices, 2000 acEve projects, single monolithic code tree with mixed languages, 20+ code changes per minute, 50,000 builds per day, 50 million test cases run per day….
18
![Page 19: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/19.jpg)
Panel Highlights: John Penix (2)
• Problems he cares about… – How do developers work and collaborate? – Team producEvity vs. individual producEvity
• All MS students should know how to do a user study
• Faculty advice: Maintain contact with your students (good links to industry)
19
![Page 20: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/20.jpg)
Panel Highlights: Lionel Briand, Simula Research Labs • PercepEons from industry: – Disconnect, engage researchers in industry – Scalability (heurisEcs versus exact methods) – Applicability (context factors, constraints), realisEc condiEons (human factors)
– Fundamental research quesEons haven’t changed… – Relevant empirical findings? • (Model based) tesEng and empirical research at MicrosoM, InspecEons
20
![Page 21: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/21.jpg)
Panel Highlights: Lionel Briand (2)
21
![Page 22: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/22.jpg)
• So you want to marry an industrial? • Success comes at a high price! – Huge Eme commitment, opportunity cost – You become a development resource, for beSer and for worse
– Effort in developing trust, what you do maXers to them, problems of today
• Industry balances strategies and considers ROI across porqolios – Do you add risk? – Evidence that customers can use it, want it, will pay for it?
Panel Highlights: Peri Tarr, IBM Research
22
![Page 23: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/23.jpg)
Panel Highlights: Peri Tarr (2)
23
![Page 24: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/24.jpg)
• Find someone with a forward looking perspecEve, Build strong, trusEng long term partner relaEonships
Panel Highlights: Peri Tarr (3)
24
![Page 25: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/25.jpg)
Panel Highlights: Tatsuhiro Nishioka, Toshiba
25
![Page 26: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/26.jpg)
Panel Highlights: Wolfram Schulte, Microso7
26
![Page 27: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/27.jpg)
Panel Highlights: Wolfram Schulte (2)
27
![Page 28: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/28.jpg)
Discussion Points
• SoMware development is a wicked problem… how to compete with snake oil salesmen with our modest soluEons?
• How to deal with kinds of evidence accepted… many kinds of important soMware dev problems are not amenable to controlled experimentaEon?
• How to become a beXer storyteller? How to deal with the apparent disconnect?
• How to do industrially relevant research without hurEng one’s academic career?
28
![Page 29: Research industry panel review](https://reader033.vdocuments.net/reader033/viewer/2022042714/554eea95b4c905d1158b53c6/html5/thumbnails/29.jpg)
ICSE Panel Slides
hXp://catenary.wordpress.com/2011/06/08/icse-‐2011-‐panel-‐slides-‐and-‐recap/
29