using ai to model quality attribute tradeoffs
DESCRIPTION
Talk from workshop on AI in requirements engineeringTRANSCRIPT
© 2014 Carnegie Mellon University
Using AI to Model Quality Attribute Tradeoffs
Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 !Neil Ernst and Ian Gorton Aug 25 2014
1
‹#›Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
NO WARRANTY
THIS MATERIAL OF CARNEGIE MELLON UNIVERSITY AND ITS SOFTWARE ENGINEERING INSTITUTE IS FURNISHED ON AN “AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
Use of any trademarks in this presentation is not intended in any way to infringe on the rights of the trademark holder.
This Presentation may be reproduced in its entirety, without modification, and freely distributed in written or electronic form without requesting formal permission. Permission is required for any other use. Requests for permission should be directed to the Software Engineering Institute at [email protected].
This work was created in the performance of Federal Government Contract Number FA8721-05-C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The Government of the United States has a royalty-free government-purpose license to use, duplicate, or disclose the work, in whole or in part and in any manner, and to have or permit others to do so, for government purposes pursuant to the copyright license under the clause at 252.227-7013.
2
Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
AI and the Requirements Problem
AI techniques
W
W, S ⊢ R
Machine learning Planning Reasoning NLP …
3
problem-rich
tool-rich
S
R
Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
Challenge 1: Scale
• People forget software, while wonderfully flexible, is subject to: • hardware limits (are you on a laptop, cluster, cloud?) • complexity limits (where is your problem in the complexity zoo?) • human limits (how many things can we understand)
• Abductive reasoning is conceptually beautiful yet intractable in reality • ATMS a wonderful framework that never went anywhere • Machine learning depends on large N for your problem
4
Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
Challenge 2: RE Knowledge
• Is RE solving the same problems as AI? • Google, translate “Min svävare är full med ål” into English not the
same as “find me a combination of services that will serve this user’s request”
• Planning a robot’s trip across the city is not the same as ‘planning’ for the design of a system.
• E.g. “there is now service X” doesn’t mean we should abandon service X-1.
5
Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
Challenge 3: Acquisition
• the Acquisition Bottleneck was a term from the expert systems era: • impossible to get what was in an MD’s head into the computer • and forget about doing it repeatably • (how is Cyc coming along anyway?)
• question: how many requirements are necessary for a complete model?
• define the context • how do we deal with changes? • need plain text for understandability to the humans (the UI
problem) but formal(ish) representations to do cool stuff on computers.
6
Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
The Tactics Wiki
Approach: encode the interesting parts machine readably • we have enough experience with
some aspects to do this with good confidence.
• tactics (or patterns) capture reusable specifications (for some rough definition of spec)
Approach: Encode the context with yes/no and location data • e.g. where are you implementing
Heartbeat?
7
‹#›Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
8
‹#›Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
9
‹#›Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
10
‹#›Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
11
‹#›Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
12
Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
So What?
Questions are contextual yet comparable E.g. for comparing security implementations in different companies Supports formal reasoning (Sparql or DL-like tools → see previous talk!) Still same 3 challenges but: manage scale with abstraction manage acquisition with templates manage RE knowledge with QA focus Help human understand while producing machine readable text. Stop annoying people.
13
Modeling QA Tradeoffs Neil Ernst, Aug 2014 © 2014 Carnegie Mellon University
Contact Information Slide
Neil A Ernst Member of Technical Staff Architecture Practices Telephone: +1 412-268-5800 Email: [email protected]
Mail Software Engineering Institute Customer Relations 4500 Fifth Avenue Pittsburgh, PA 15213-2612 USA
Web www.sei.cmu.edu www.sei.cmu.edu/contact.cfm !
Customer Relations Email: [email protected] Telephone: +1 412-268-5800 SEI Phone: +1 412-268-5800 SEI Fax: +1 412-268-6257
14