multimodal intelligent interactive development environment
DESCRIPTION
Multimodal Intelligent Interactive Development Environment. Application Programming Interface 8/12/2009. System Overview. Model & Knowledge Management. Get context. MIIDE Whiteboard (MIDOS). Context. Task Models. Device Models. Preferences& constraints. Model Reasoner. Evaluate. - PowerPoint PPT PresentationTRANSCRIPT
BAE SYSTEMS PROPRIETARY INFORMATIONThe information contained in this document is the property of BAE Systems, and further dissemination is prohibited without the written permission of BAE Systems.
UNCLASSIFIED
UNCLASSIFIED
Multimodal Intelligent Interactive Multimodal Intelligent Interactive Development EnvironmentDevelopment EnvironmentApplication Programming Interface8/12/2009
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
MIIDEWhiteboard
(MIDOS)
PAMS LTAE
ModelReasoner
Get context
Evaluate
Preferences& constraints
Model & Knowledge
Management
Deploy and Simulate
TaskModels
Results & Rationale
Context
Validation results
Simulator
DeviceModels
InferenceRules
System OverviewSystem Overview
Results & rationale
Context Models
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
• MIDOS – user interaction core
• MKM – Models and tools• Continuous two way
interaction between MIDOS and MKM
• Domain specific plug-insMIDOS
Domain independent
Domain specific
Domain perspective
Model & Knowledge
Management
Model Compiler
MIIDEDomain meta model
The model
Simulator/Validator
Generated Artifact
User models
Model
MIIDE Plug-In ArchitectureMIIDE Plug-In Architecture
Generated
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
ChallengesChallenges
• Incremental interaction between MIDOS and MKM to guide the user• Intelligently narrowing down the future options• Allowing individual user to develop domain dialects for interacting with
MIDOS• Evolving the domain meta model
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
Simulator
MIIDEWhiteboard
(MIDOS)
PAMS LTAE
ModelReasoner
SDR Simulator(Testsys)
Ontology.xsd
Deplo
y
getOntology()
deploy()
evaluate()
S-expression
evaluate()
Model & Knowledge
Management
deploy()
Models
Component APIComponent API
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
sd getOntology
MIDOS Model & KnowledgeManagement
MIDOS User
StartStart
Intialize()
getOntology()
:ServiceDescription, CapabilityGroup
How may I help you?()
getOntology()getOntology()
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
• An instance of a ServiceDescription defines all of the domain objects to be used by MIDOS.
• A set of components defines a system that will provide one or more capabilities.
class Serv ice Diagram
Serv iceDescription
- serviceName: string
Env ironmentSpec
- environmentName: string- value: string
ComponentSpec
- componentName: string
ResourceCost
- amount: string- resource: ResourceSpec
ReliabilityInEnv ironment
- reliability: double- environmentCondition: EnvironmentSpec
CapabilitySpec
- defaultValue: boolean- capabilityName: string- value: string- description: string
Default assumes 100% reliability, if condition not present
ResourceSpec
- resourceName: string
0..*
typesOfComponents
0..*
environmentConstraints
0..*
resourcesUsed
0..*
typesOfCapabilities
0..*
typesOfEnvironments
0..*
typesOfResources
0..*
resourcesSupplied
getOntology() resultsgetOntology() results
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
ServiceDescription
serviceName SDR
List<CapabilitySpec>
capabilityName value
Video Recording Standard
Video Recording High-def
Onboard Storage None
Onboard Storage Low capacity
Onboard Storage High capacity
Audio Recon Yes
Audio Recon No
Weapon Delivery Yes
Weapon Delivery No
List<EnvironmentSpec>
environmentName value
Weather Clear
Weather Cloudy
Weather Precipitation
Weather Heavy Precipitation
Terrain Mountain
Terrain Valley
Terrain Plain
Terrain Ocean
List<ResourceSpec>
resourceName
Power
Memory
Weight
typesOfCapabilities
typesOfResources
typesOfEnvironments
Service DescriptionService Description
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
ServiceDescription
serviceName SDR
List<ReliabilityInEnvironment>
reliability environmentCondition
50% Precipitation
environmentConstraints
List<ResourceCost>
resource amount
Weight 100 lbs
Memory 96 MB
List<ComponentSpec>
componentName Predator
componentName CameraList<ReliabilityInEnvironment>
Reliability environmentCondition
25% Precipitation
0% Heavy Precipitation
List<ResourceCost>
resource amount
Weight 15 lbs
Power 25 Watts
environmentConstraints
resourcesSupplied
resourcesUsed
typesOfComponents
Service Description – ComponentsService Description – Components
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
Mission Type
Recon KineticWeaponDelivery: YesPrecision Guidance: Yes
VideoAudioAudio Recon: YesOnboard Storage: Yes
Close Air SupportVideo Recon: StandardRetransmission: Yes
Destroy TargetTrack: Yes
High-DefinitionHD Video Recon: YesHD Onboard Storage: Yes
StandardVideo Recon: YesRetransmission: Yes
class Serv ice Diagram
CapabilityGroup
- name: string
Capability
- name: string- value: string
0..*
capabilitiesInGroup
0..*subGroup
getOntology() results: CapabilityGroupgetOntology() results: CapabilityGroup
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
Evaluate will be called each time the user selects or removes a capability (either implicitly or explicitly). This will return to MIDOS 0..n valid configurations with reasons, which will allow MIDOS to tailor the information it presents to the user.
evaluate()evaluate()
sd ev aluate
MIDOS Model & KnowledgeManagement
Model Reasoner
MIDOS User
selectCapability()
evaluate(Request, Context)
evaluate(ServiceDescription,Configurations, Request, Context)
:ReasoningResults
update()
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
Each request has a set of required components and capabilities and a set of weighted capability preferences.
class Serv ice Diagram
Context
- serviceName: string
Resource
- available: string- resourceCost: double- resourceType: ResourceSpec
resourceCost establishes a relative cost across different resources; can be used by reasoners to minimize the cost of resources used to implement a given capabil ity set
:Env ironmentSpec
0..*
resourcesenvironment
0..*
The context can contain values for each resource (Power, Weight, etc.) and values for the environment (Weather: Cloudy, Terrain: Mountain, etc.)
class Serv ice Diagram
Request
- requestName: string- serviceName: string- description: string
CapabilityPreference
- leftBetterThanRightBy: double
Capability
- name: string- value: string
Capability
- name: string- value: string
Capability
- name: string- value: string
Component
- name: string
1..*
left
0..*
capabilityPreferences
0..*
requiredCapabilities
1..*
right
0..*
requiredComponents
evaluate() input: Request and Contextevaluate() input: Request and Context
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
The ReasoningResults returned from the evaluate method call will contain an ordered list of 0..n results and the explanation of why no configuration was selected or a summary of why the n-configurations were selected.
Each result will contain:•validConfiguration: This configuration will contain all of the required capabilities and components and may contain some of the preferred capabilities.•comparisons: 0..n comparisons between the validConfiguration and other configurations that details the difference between the two and why the validConfiguration ranks higher.
evaluate() resultsevaluate() resultsclass Serv ice Diagram
Configuration
- configName: string- serviceName: string
Capability
- name: string- value: string
:ComponentSpec
ReasoningResults
- explanation: string
Result
Rationale
- difference: string- reason: string
1..*
capabili ties components
1..*
1
compareTo
1
validConfiguration
0..*
comparisons
0..*
results
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
The user requests to add image filtering and sets the priority of the capabilities.
Storyboard ExampleStoryboard Example
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
List<Capability>
name value
Predator Yes
Audio Recon Yes
Video Recon Standard
Non-LOS Retransmission
Yes
Precision Guidance Yes
List<Component>
name
Camera
Request
requestName Request1
serviceName SDR
List<CapabilityPreference>
leftBetterThanRightBy 10
List<Capability>
name value
Image Filtering Yes
List<Capability>
name value
Image Filtering No
capabilityPreferences
left right
requiredCapabilitiesrequiredComponents
Example RequestExample Request
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
List<EnvironmentSpec>
name value
Weather Precipitation
Terrain Mountain
List<Resource>
resourceType resourceCost available
Power 2 20 Watts
Other resource constraints will be determined by what is supplied by the selected components in a configuration. In this example, the system might know that an additional 20 Watts is being provided by solar panels mounted on the Predator.
Context
serviceName SDR
resources environment
Example ContextExample Context
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
capabilities components reason
Predator: Yes
Audio Recon: Yes
Video Recon: Standard
Non-LOS Retransmission: Yes
Precision Guidance: Yes
Hardware: Predator, GPS Device, Camera, Microphone, High-Power Modem
Software: Flight Computer Interface, Internal Network Stack, Audio Acquisition, Video Acquisition, SNMP Agent
Result 1:
Result 2:
capabilities components reason
Predator: Yes
Audio Recon: Yes
Video Recon: Standard
Non-LOS Retransmission: Yes
Precision Guidance: Yes
Hardware: Predator, GPS Device, Camera, Microphone, High-Power Modem
Software: Flight Computer Interface, Internal Network Stack, Audio Acquisition, Video Acquisition
Uses less resources
Reasoning ResultsReasoning Results
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
Deploy can be called multiple times to test different configurations. Each call will present a simulation to the user and run generated unit tests that return if the configuration is valid.
sd deploy
MIDOS Model & KnowledgeManagement
Simulator
MIDOS User
deploy(Configuration)
simulate()
simulation()
:isValidConfiguration
deploy()deploy()
BAE SYSTEMS PROPRIETARY INFORMATION
UNCLASSIFIED
UNCLASSIFIED
• Add a new CapabilityGroup• Reason about preferences at CapabilityGroup level• Allow the user to add a new capability and use the PAMS DTA for SDR to
dynamically generate valid configurations• Use the DESERT domain constraints to provide better reasons why
configurations are not valid• Drive dialog from set of potential configurations
Possible ExtensionsPossible Extensions