network centric

RAND

OFTASDC3I

EBR

A Conceptual Frameworkfor

Network Centric Warfare

Workshop on Network Centric Warfareand

Network Enabled CapabilitiesDecember 17-19, 2002

Ongoing Research Sponsored by OFT and ASD(C3I)

8/3/2004 Slide 2RAND

OFTASDC3I

EBR

Agenda

• Informing Transformation

• The NCW Framework Initiative

• The NCW Framework

• Elements of the Force

• NCW Measures, Attributes, and Metrics

• Case Study: Air-to-Air Combat

• Summary and Next Steps


OFTASDC3I

EBR

Informing Transformation

• NCW concepts are the military embodiment of Information Ageconcepts/technologies

• Early insights emerging – fundamental questions remain- Does NCW help make the force agile?- What is the best way to command and control a network-centric force?- How do we create a network-centric force?- How can we measure progress toward achieving a network-centric force?

• Requires a new theory and supporting body of knowledge- What experiments should we do?- What research is needed?

• Requires mechanism for development and application of theory byDoD and its allies

• Begins with a new conceptual framework and assessmentmethodology/tools

DoD transformation is, at its core, a military adaptation to the Information Age


OFTASDC3I

EBR

DoD Priorities and Goals

• Priorities of the Office of Force Transformation inDoD

– “Get the metrics right and applied enterprise wide”

• Desired Status in 5 Years Time– “Get the metrics right…”

• Establish conceptual framework accompanied by mature theory andunderstanding of NCW

– “…And applied enterprise wide”• Sufficient number of organizations throughout Government,

academia, and industry with knowledge of the NCW Conceptualframework and the ability to apply it to solve real world problems


OFTASDC3I

EBR

NCWFocusedResearch

NCWKnowledge

Base

NCWFocused

Experiments

NCW Theory(Hypotheses)

ConceptualFramework

Metrics

Board of Directors

Awareness

Education

Consulting

•Analysis•Experiments•Exercises•Case Studies•T & E•Etc.

EnterpriseApplications

Relationships

Code of Best Practice

Tools

Methodology

The NCW Framework InitiativeKey To Developing and Applying NCW Theory Across DoD Enterprise


OFTASDC3I

EBR

Nature of NCW Conceptual Framework

• Based on current tenets of NCW– Potential new sources of combat power

• Includes key concepts and their– Measures– Attributes– Metrics– Relationships

• Provides basis for quantitativeexploration/assessment

– NCW hypotheses– Investment strategies– Other DOTML-PF related issues


OFTASDC3I

EBR

Info structure

Sensor NettingData Fusion

Information Management

Vastly Improved AwarenessShared Awareness

Virtual CollaborationVirtual Organizations

Substitution of Info. forPeople and Material

Self - Synchronization

Increased TempoIncreased Responsiveness

Lower RisksLower Costs

Higher Profits

Enabler

Process forGenerating Awareness

Enabler

Process forExploiting

Awareness

Results

“ The Entry Fee”

“The Bottom Line”(Measurable)

Info structure

Sensor NettingData Fusion

Information Management

Vastly Improved AwarenessShared Awareness

Virtual CollaborationVirtual Organizations

Substitution of Info. forPeople and Material

Self - Synchronization

Increased TempoIncreased Responsiveness

Lower RisksLower Costs

Higher Profits

Enabler

Process forGenerating Awareness

Enabler

Process forExploiting

Awareness

Results

Figure 6. The Network Centric Enterprise

P 36, Network Centric Warfare: Developing and LeveragingInformation Superiority. CCRP. 1999

“ The Entry Fee”

“The Bottom Line”(Measurable)

NCW Framework Evolution

• A robustly networked force improves information sharing

• Information sharing and collaboration enhances the quality of information and shared situational awareness

• Shared situational awareness enables collaboration and self-synchronization, and enhances sustainability and speed of command

• These in turn dramatically increase mission effectiveness

Tenets of NCW (DoD Report to Congress on Network Centric Warfare):

NCW Conceptual Framework (2002)NCW Foundation (1999)

AwarenessUnderstanding

Degree of Decision/ Plan Synchronization

Degree of Actions/ Entities Synchronized

Degree of Effectiveness/ Agility

Degree of Information “Share-ability”

Degree of Networking

Force

Quality of Individual Information Degree of Shared Information

Quality of Individual SensemakingShared Awareness

Shared Understanding

Degree of Shared Sensemaking

Operating Environments

Quality of Organic Information

Information Sources C2 Effectors

Value Added

Services

Quality of Collaborative DecisionsQuality of Individual Decisions

Physical Domain

Social Domain

Information Domain

Cognitive Domain

Qualityof

Inter-actions


OFTASDC3I

EBR

NCW Conceptual Framework













ForceInformation Sources C2 Effectors

Value Added Services


Quality of

Inter-actions


OFTASDC3I

EBR

NCW Traverses Four Key Domains

Physical Domainwhere strike, protect, and maneuver take place across

different environments

Information Domainwhere information is created, manipulated and shared

Cognitive Domainwhere perceptions, awareness, beliefs, and values reside

and where, as a result of sensemaking, decisions are made

Social Domainwhere force entities interact


OFTASDC3I

EBR







Force







C2 EffectorsValue Added

Services


Physical Domain

Social Domain

Information Domain

Cognitive Domain

Quality of

Inter-actions

Information Sources



OFTASDC3I

EBR

Command & Control

People

Key Elements: Nodes and Networks

Sensors Networks

Effectors (Shooters)

Force


OFTASDC3I

EBR

Force

Mission Capability Packages

Roles/ Functions

Measures (Exogenousto the NCW framework)

Elements(Network, Nodes)

People, Platforms, Facilities, Units, Networks, ...

Value added Services C2Information

SourcesEffectors

• Phenomenology• Coverage• Persistence• Performance• Agility

• Service• Capability• Capacity• Quality of Service• Agility

Embedded in the NCW conceptualframework

D O T M L P F

• Effects• Coverage• Persistence• Survivability• Agility

…n1

Measures for Key Elements


OFTASDC3I

EBR

Information Sources Value Added Services C2 Effectors

Force


Consistency

TimelinessPrecision

Currency

Accuracy

CompletenessCorrectness

Relevance

Objective Measures Fitness for Use

Quality of Individual Information

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Objective Measures Fitness for Use Degree of Information “Share-ability”

Quantity of Retrievable Info

Quantity of Posted Info

Ease of Use

Node Assurance

Collaboration Support

P&R Capability Support

Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach

Degree of NetworkingNetwork Net Ready Nodes

Qualityof

Interactions

NCW Conceptual Framework:Summary of Attributes (1)


OFTASDC3I

EBR

Degree of Networking:Network

Degree to which network can maintain quality of service in response toenvironmental changes (incorporates robustness, responsiveness, flexibility,innovativeness and adaptation)

Network Agility

Extent to which network provides services that facilitate the assurance ofinformation in the areas of privacy, availability, integrity, authenticity, andnonrepudiation

Network Assurance

Ability of network to provide a variety of communications and storageservices

Quality of Service

The degree to which force entities can connect and communicateReach

DefinitionAttribute

The extent to which force entities are interconnected


OFTASDC3I

EBR

Degree of Networking:Network

See next slideNetwork Agility

Categorical rating from “highly secure” to “not secure”

(estimated from assessment of network’s installed security software,hardware, and usage policies)

Network Assurance

Vector of performance metrics, including average bandwidth provided(available and bottleneck), packet delay, delay jitter, and data loss

Quality of Service

Percent of nodes that can communicate in desired access modes, informationformats, and applications

Reach

MetricsAttribute

The extent to which force entities are interconnected


OFTASDC3I

EBR

Degree of Networking:Network Agility

The timeliness of the response to an environmental change (baseline leveldetermined by SME, simulation, analysis, empirical analysis, etc.)

Responsiveness

Number and timeliness of changes to network structure and processes (baselinedetermined by SME, simulation, analysis, empirical analysis, etc.)

Adaptiveness

Number of options for responding to an environmental change

Compatibility of different responses (0=not compatible, 1=fully compatible;determined by SME, simulation, analysis, empirical analysis, etc.)

Flexibility

Number of novel responses developed and implemented(baseline determined by SME, simulation, analysis, empirical analysis, etc.)

Innovativeness

Number of differing conditions/environments over which network is capable ofoperating at a given level of effectiveness (baseline level determined by SME,simulation, analysis, empirical analysis, etc.)

Effectiveness of network across varying levels of attack/degradation (baseline leveldetermined by SME, simulation, analysis, empirical analysis, etc.)

Number of tasks/missions which the network is capable of operating at a givenlevel of effectiveness (baseline level determined by SME, simulation, analysis,empirical analysis, etc.)

Robustness

MetricsAttribute


OFTASDC3I

EBR


Quantity of Retrievable InfoQuantity of Posted InfoEase of Use


Consistency

TimelinessPrecision

Currency

Accuracy


Relevance

Objective MeasuresFitness for Use

Relevance

Completeness

Timeliness

Accuracy

Extent

Degree of Shared Information

Currency

Consistency

Correctness

Precision

Quality

Objective Measures

Quality of Individual Sensemaking: Awareness

Currency

Consistency

Correctness

Precision

Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for UseObjective Measures

Completeness

Timeliness

Accuracy

Extent

Degree of Shared Sensemaking: Shared Awareness

Currency

Consistency

Correctness

Precision

Quality

Objective Measures

Uncertainty


Individual Characteristics

Organizational Characteristics

Organizational & Individual Behavior

Quality of Interactions

QualityDepth

Breadth

Agility

Adaptability

Flexible

IntensitySelectivity

Mode

Latency

Response

Trust

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident Confidence

Competence

Risk Prop

Diversity

Competence

Confidence

Hardness

Trust

Size

Permanence

Autonomy Structure

Interdepend

Synchronization

Efficiency

T vs. T

Cooperation

Engagement


OFTASDC3I

EBR

IndividualCharacteristics

•Risk Propensity•Competence•Trust•Organizational Identification•Confidence

Organizational andIndividual Behaviors

•Cooperation•Efficiency•Synchronization•Engagement•Team vs. Task Balance

Quality of Interactions:Dimensions and Attributes

The focus of interaction: share information, develop and share awareness, develop and share understandings, make decisions


•Depth•Breadth•Intensity•Agility

OrganizationalCharacteristics

•Risk Propensity•Competence•Trust•Confidence•More ..


OFTASDC3I

EBR

The persistence of the exchange among members (continuous to episodic)Continuity

Degree to which all senses are involved (ranges from face to face with data + voice tovoice or data only)

Mode

The quantity of information, awareness, understandings, and/or decisions that are thefocus of interactions

Quantity

Type of interaction: synchronous or asynchronous in time and spaceSynchronicity

The ability to reach a selected sub-setSelectivity

Robustness, Flexibility, Responsiveness, Innovativeness, and AdaptabilityAgility

The time lag of interactionsLatency

Measures that describe the pace and completeness of interactionsIntensity

The number of members that participate in the interactionsReach

Measures that describe the force entities that interactBreadth

The quality of information, awareness, understandings, and/or decisions that are the focusof the interactions

Quality

Measures that describe the nature of the substance of interactionsDepth

DefinitionAttribute

Quality of InteractionsTop Level Attributes


OFTASDC3I

EBR


• Shared Awareness- Those aspects of individualviews of the battle space that are shared across two ormore force entities/organizational members

• Shared Understanding- Those recognitions,including patterns, cause and effect relationships,dynamic futures, and opportunities and risks, that areshared across two or more force entities/organizationalmembers


OFTASDC3I

EBR

Degree of Shared Sensemaking:Shared Awareness

Extent to which shared awareness is consistent with ground truthCorrectness

Extent to which shared awareness is consistent within and across CoIConsistency

Time lag of shared awarenessCurrency

Level of granularity of shared awarenessPrecision

Measures quality in reference to criteria that are determined by the situationQuality

Extent to which relevant shared awareness is obtainedCompleteness

Appropriateness of precision of shared awareness for a particular useAccuracy

Proportion of shared awareness obtained that is related to task at handRelevance

Extent to which currency of shared awareness is suitable to its useTimeliness

Subjective assessment of confidence in shared awarenessUncertainty

Proportion of awareness in common across force entities, within and across communitiesof interest (CoI)

Proportion of force entities that share a given awareness

Extent

Measures quality in reference to criteria that are independent of the situationObjective Measures

DefinitionAttribute


OFTASDC3I

EBR


Currency

Consistency

Correctness

Precision Uncertainty

Timeliness

Relevance

Accuracy

Completeness


Completeness

Timeliness

Accuracy

Extent


Currency

Consistency

Correctness

Precision

Quality

Objective Measures

Uncertainty

Quality of Individual Sensemaking: Understanding

Currency

Consistency

Correctness

Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Precision


Completeness

Timeliness

Accuracy

Extent

Degree of Shared Sensemaking: Shared Understanding

Currency

Consistency

Correctness

Precision

Quality

Objective Measures

Uncertainty

Quality of Individual DecisionsFitness for Use

Timeliness

Relevance

Accuracy

Completeness

Appropriateness

UncertaintyInnovativeness

Agility

Responsiveness

Flexibility

Robustness

Adaptability

Objective Measures

Mode of D. M.

Precision

Currency

Consistency

Risk Propensity

Quality of Collaborative DecisionsFitness for Use

Innovativeness

AgilityObjective Measures

Mode of D. M.

Precision

Currency

Consistency

Extent

Timeliness

Relevance

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

AdaptabilityRisk Propensity



Organizational & Individual Behavior


QualityDepthBreadth

Agility

Intensity

Response

Trust

Interdepend

Synchronization

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Engage



OFTASDC3I

EBR

Quality of Collaborative Decisions I

Extent of risk aversionRisk Propensity

Type of collaborative decision making structure utilized (authoritative decision making, consensus building, majority rule,etc.)

Mode of Decision Making

Proportion of force entities that reach a collaborative decisionExtent

Extent to which decisions are consistent with existing shared understanding, command intent and shared team valuesAppropriateness

Inter-subjective assessment of confidence in decisionsUncertainty

Measures quality in reference to criteria that are determined by the situationFitness for Use Measures

Extent to which relevant decisions encompass the necessary:

•Depth: range of actions and contingencies included

•Breadth: range of force elements included

•Time: range of time horizons included

Completeness


Proportion decisions that are important to the accomplishment of the task at handRelevance

Appropriateness of precision of decisions for a particular useAccuracy

Extent to which currency of decision making is suitable to its useTimeliness

Level of granularity of decisionsPrecision

Extent to which decisions are in agreement across force entities, within and across CoIConsistency

Time lag of decisionsCurrency

DefinitionAttribute


OFTASDC3I

EBR

Quality of Collaborative Decisions II

Degree to which collaborative decision facilitates force entities’ ability to alter thedecision, decision making participants and/or decision making process and implementappropriate modifications

Adaptability

Agility

Degree to which collaborative decision reflects novel ways to perform known tasksand/or develops new ways of doing novel tasks

Innovativeness

Degree to which collaborative decision allows force entities to maintain flexibility (i.e.,incorporates multiple ways of succeeding)

Flexibility

Degree to which collaborative decision is relevant and timelyResponsiveness

Degree to which collaborative decision is dominant across a range of situations anddegradation conditions

Robustness

DefinitionAttribute


OFTASDC3I

EBR

Qualityof Interactions

Quality of Collaborative DecisionsFitness for Use

Objective Measures

Mode of D. M.

TimelinessPrecision

Innovativeness

Agility

RelevanceCurrency

Consistency

Extent

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

Adaptability



Synchronized Decisions/Plans


Synchronized Actions Synchronized Entities


Achievement of Objectives Agility Timeliness Efficiency



Objective Measures

Mode of D. M.

TimelinessPrecision

Innovativeness

Agility

RelevanceCurrency

ConsistencyAccuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

AdaptabilityRisk Propensity Risk Propensity


OFTASDC3I

EBR

Degree of Decisions andActions Synchronized

Degree of Decisions / Plans Synchronized

• Synchronized Decisions/Plans: Proportion ofdecisions/plans that are conflicted, de-conflicted or synergistic

Degree of Actions / Entities Synchronized

• Synchronized Actions: Proportion of actions that areconflicted, de-conflicted or synergistic

• Synchronized Entities: Proportion of force entities whosepositions are conflicted, de-conflicted, or synergistic


OFTASDC3I

EBR

Degree of Effectiveness / Agility

Total cost of achieving objectiveEfficiency

The degree to which force entities were robust, flexible, responsive,innovative, and adaptable

Agility

Time required to achieve objectiveTime

Degree to which Military/Strategic/ Political/ Social/ Economic/Diplomatic objectives were achieved

Achievement ofObjectives

DefinitionAttribute


OFTASDC3I

EBR

Framework: Attributes and Metrics







Force







C2 EffectorsValue Added

Services


Quality of

Inter-actions

Information Sources

Parts of frameworkfor which attributesand metrics havebeen developed

Parts of framework forwhich attributes havebeen developed


OFTASDC3I

EBR

Some Issues (1)• Social Domain

– Is this really a domain? If so, what is its relationship to the cognitivedomain?

• Survivability– Are vulnerability and potential degradation of networked forces adequately

accounted for in the framework?

• Synchronization– Should this measure explicitly account for asynchronization?

• Coherence– Should this be a separate measure? If so, how does it relate to

synchronization?

• Force Cohesion– Is this an important indicator of mission success? How does it relate to

degree of interaction?


OFTASDC3I

EBR

Some Issues (2)• Quality of Interactions

– Are some of the attributes exogenous variables?

• Integration– Should this be an explicit measure?– How does it relate to degree of interaction?

• Agility– Is this measure adequately represented in the framework?– Should it be more systemic?

• Mission Capability Packages– Should the relationship between exogenous variables and DOTML-PF

be more explicit?

• Operating Environment– Is its relationship to other measures proper?– What are the appropriate attributes?


OFTASDC3I

EBR

Agenda





• NCW Measures, Attributes and Metrics




OFTASDC3I

EBR

The JTIDS Operational Special ProjectResults from 12,000 sorties in Air-to-Air Combat

•Conditions-AWACS with fighter aircraft-Range from 2 on 4 aircraft up to 8 on 16 aircraft missions-Day and night engagements-Voice only vs. voice + Link 16

•Results (Kill Ratio, X:1)

9.40 3.62Night

8.11 3.10Day

Voice + Link 16Voice Only


OFTASDC3I

EBR

Air-to-Air Scenario ExemplarFour-on-Four Engagement

Blue11, 12

Blue13, 14

AWACS

Red 1, 2

Red 3, 4


OFTASDC3I

EBR

Framework: Data Available for Case Study













Parts of Frameworkfor which reasonabledata is available

Parts of Frameworkfor which data iscurrently unavailable

ForceInformation Sources C2 Effectors

Value Added Services


Quality of

Inter-actions


OFTASDC3I

EBR

Force

Mission Capability Packages

Roles

RelevantAttributes

Elements(Network, Nodes)

Value added Services C2Information

SourcesEffectors

• Coverage• Persistence• Performance

• Capability• Capacity• Quality of Service

Embedded in the NCW conceptualframework

• Target Destruction

Voice Only(MCP #1)

Voice NetworkAWACS Air Craft

Functions • Detect/ID targets

•Fuse data•ID info•Distribute Info.

•Assign aircraft totargets•Coordinateengagements

•Kill Targets

Voice + Link 16(MCP #2)

Data +Network


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach



Force



Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance



NetworkNet Ready Nodes

Qualityof

Interactions

Influence of the Force on Quality ofOrganic Information


OFTASDC3I

EBR

Sensor coverage - Field of regard - Instantaneous field of view - Area coverage rate - Revisit rate (e.g. AWACS 10 sec scan) - RangeProbability of detection

-Fn of RCS-Fn of doppler

Probability of false alarmProbability of classificationSighting location errorSighting velocity errorRadar processor track precision

Computing Quality of Organic Information

Information Sources

Force


Consistency

TimelinessPrecision

Currency

Accuracy


Relevance


AWACs

AMTI radarIFFN

NCTR#1Etc.

Blue12 Blue14Blue11 Blue13

AMTI radarIFFN

NCTR#1Etc.

AMTI radarIFFN

NCTR#1Etc.

Exogenous variables• Environmentalconditions• Doctrine

AMTI radarIFFN

NCTR#1Etc.

AMTI radarIFFN

NCTR#1Etc.

f(…)


OFTASDC3I

EBR

Quality of Organic Information:Threat Tracks

TTTT QQQQ 4321

Blue11, 12

0

0.25

TTTT QQQQ 4321

Blue13, 14

0

0.25

0.5

0.75

1

TTTT QQQQ 4321

TQ 1

TQ 2

TQ 3

TQ 4

Completeness: Detection

Correctness: ID

Correctness: Location

Correctness: Velocity

0

0.25

0.5

0.75

1


OFTASDC3I

EBR

Comparing MCPs Using Summary MetricsQuality of Organic Information

Degree of SharedInformation

1.0

*0.28

Overall average over information qualitydimensions and package members

Quality of IndividualInformation

Degree ofNetworking

Degree ofInformation

“Share-ability”

Quality ofIndividualDecisionsKill Ratio

(Synchronization / Effectiveness)

Quality ofOrganic Info

Quality ofIndividual

Sensemaking

**

VoiceVoice + Link 16


OFTASDC3I

EBR

Comparing MCPs Using Summary MetricsSynchronization and Effectiveness


**




Degree ofNetworking


“Share-ability”





Sensemaking1.0

0.28

0.51.00.08

1.0

0.4

0.91

3.10:1

8.11:1

1.0

0.22

0.450.91

0.45

0.91


OFTASDC3I

EBR

Quality of Sensemaking & Decisions Relative Speed and Competitive Advantage

Hypotheses:• Information sharing via Voice + Link 16 leads to less time necessary to gather critical

information, which results in more time available for flight lead to develop sensemakingand make decisions

• Information sharing via Voice + Link 16 leads to less time necessary for wingman togather and monitor critical information, which results in opportunities for wingman tospend time sensemaking and making decisions

Time

B11 (Flight lead)

Voice Only

Link-16 +Voice

Awareness

Information

Information

AwarenessInfo

Understanding Awareness Decisions

Understanding Decisions

Voice Only

Link-16 +Voice

B12 (Wingman)

AwarenessInfo Understanding Decisions


OFTASDC3I

EBR

Degree of Synchronization and EffectivenessReported Tactical Improvements Enabled by Voice + Link 16

• Voice + Link 16 allows greatly increased information sharing, leading to nearly-comprehensive awareness and understanding of air-to-air battlespace

• Greater understanding allows for use of four types of “high-awareness” tacticsthat lead to major increases in combat effectiveness

1. Increased numbers of engagements inthe same time period

2. Employment of the wingman ascombatant rather than defensive patroller

3. Advance vectoring to engage red A/Csfrom position of maximum advantage

4. Employment of cooperative formations totrap and destroy red A/Cs

Time

Flight lead

Wingman


OFTASDC3I

EBR

Areas that Require AdditionalAttention for Air-to-Air Case Study

• Data describing cognitive and social behavior– Quality of interactions

– Sensemaking / Decision making

• Impact of non-material changes in DOTML-PF– C2 concept (e.g., role of AWACs)

– Changes in tactics, techniques and procedures

• Impact of changes in force mix– A/C, sensor and weapon type

• Effects of scaling number of A/C– Impact on net performance

– Impact on mission effectiveness

• Accounting for dynamics over time


OFTASDC3I

EBR

Agenda





• NCW Measures, Attributes and Metrics




OFTASDC3I

EBR

Potential Next Steps

“Getting NCW Theory and Metrics Right…”– Continue to refine and evolve the framework– Complete Air-to-Air case study

• Obtain additional data and address remaining measures

– Disseminate framework and obtain additional peer review

“…And Applied Enterprise-Wide”– Engage potential users of framework to establish new

opportunities for application– Develop methodologies for applying framework in support

of transformation– Conduct broad range of case studies with key partners

Establish Board of Directors to shape priorities and ensure quality


OFTASDC3I

EBR

Closing Thoughts

• We are making progress in developing/applying the framework - Well into definition of second generation framework - However, significant issues remain

• There is growing interest in applying the framework - Wide range of potential applications - Numerous opportunities for collaboration

• Important to keep up momentum - Refine/extend framework - Identify and enable key applications

• Broad community-wide participation is critical


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach



Force

Ease of Use


Computing Quantity of Posted Info: Track Info over Voice Only

Exogenous variables:•Nodes: AWACS, F-15s•Types: 4 red tracks, 5 bluetracks, etc.•CONOPS, coding schemes,governing how to speaktrack info•Red tracks have priorities,but two strike packagesmust know each other’spositions•Track info “expires” afterten seconds

•AWACS, F-15s can transmitover radio

•One military vocoder channel

•Assumed to be 100% in abenign environment

•No adjustments (static voicebroadcasting network)

• In this scenario, quantity of postedinfo equals quantity of retrievable info,except for probability of hearing voice

f(…)

!!!!General Link 16


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach


C2 Effectors

Force

Ease of Use


Computing Quantity of Posted Info: Track Info over Link 16


•AWACS, F-15s can transmittrack info over Link 16

•One shared Link 16 network(capacity greatly exceedsnumber of info items here)


•No adjustments (static wirelessnetwork)

• In this scenario, quantity ofposted info over Link 16 equalsquantity of retrievable info

Information Sources Value Added Services

f(…)

!!!!General Voice Only


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach


Source/Sensors Value Added Info Processors C2 Effectors

Force

Ease of Use


Computing Quantity of Posted Info:Detailed Function for Posted Info

Exogenous variables:•Number of nodes•File sizes and number offiles•Variables impacting howquickly nodes can transmitpieces of information(CONOPS, coding schemes,etc.)•Policies determiningpriority for posting•Expiration age for eachtype of info objects•Maximum queue lengths

•Whether nodes can transmit tonetwork

•Posting channel numbers, types,and bandwidth (for data linksonly)

•Prob of correct transmission

•Adjustments to probability thatQoS will be delivered

f(…): Vector for number of info objects that can be posted, by object typePhase 1. For each type of info object do:• If nodes can post object, do 2. Else, Num(type) = 0• Use QoS parameters, network agility parameters, and exo variables to

determine rate at which nodes can post info items of that type. Multiply thisrate by probability of correct transmission, yielding theoretical transmissionrate.

Phase 2.• Use theoretical rates for each info type plus priority policies to determine what

fractions of postings will be of each info type.• Multiply fractions of postings times theoretical rates times expiration age to

get Num(type) for each info type.!!!!General


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach



Force

Ease of Use



•AWACS, F-15s can transmitover radio

•One military vocoder channel


•No adjustments (static voicebroadcasting network)

f(…): Vector for number of info objects that can be posted, by object typePhase 1. For each type of info object do:• AWACS, F-15s can post (do 2)• Using AF CONOPS and coding standards, AC can transmit three tracks every

ten seconds on a military coding voice channel. In this scenario, this rate isunaffected by network assurance considerations.

Phase 2.• Using AF priority policies, on average each 10-sec interval has two red tracks

and one blue track.• Since each track “lasts” for ten seconds, at most an average of two red tracks

and one blue track can be posted at any time.

Computing Quantity of Posted Info:Detailed Function for Voice Only Network

!!!!Voice

f(…): Vector for number of info objects thatcan be retrieved, by object type

• 70% of voice tracks are audible, so metricis 0.7*Quantity of Posted Info


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach



Force

Ease of Use



•AWACS, F-15s can transmittrack info over Link 16

•One shared Link 16 network(capacity greatly exceedsnumber of info items here)


•No adjustments (static wirelessnetwork)

F(…): Vector for number of info objects that can be posted, by object typePhase 1. For each type of info object, do:• AWACS, F-15s can post (do 2)• Using Link 16 capacity and AF track coding standards, rate exceeds

maximum number of tracks updated every second.Phase 2.• All tracks can be posted at least every second, so no priority policies apply.• Info on all tracks can be updated every second, so info on all nine tracks can

be posted at any given time.

Computing Quantity of Posted Info:Detailed Function for Link 16 Network

!!!!Link 16


OFTASDC3I

EBR






Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance



Computing Correctness for Individual Information: Voice Only

•Organic info assumed tobe correct, within knownmargins of error

Exogenous variables:•Track, capability, intentinformation, all instandard formats•No fusion performed•Pilots will use organicinformation in preferenceto radio-reportedinformation•Pilots have sufficienttraining to use radio

•Voice track messages onlyaudible 70% of the time

•Military vocoder replicateshearing errors (nocorrection7)

f(…)

!!!!General Link 16


OFTASDC3I

EBR






Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance



Computing Correctness for IndividualInformation: Link 16


Exogenous variables:•Track, capability, intentinformation, all instandard formats•Fusion consolidates bluetracks only•Pilots will use F-15 radarinformation in preferenceto AWACS information•Pilots have sufficienttraining to use Link 16display and radio

•Assuming Link 16 reachesblue AC with no infodegradation

•Display screen assumed tobe error free within screenresolution

f(…)

!!!!General Voice


OFTASDC3I

EBR


“Share-ability”Quantity of Retrievable Info


Quality ofOrganic

Information


Information

Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance



•Correctness of organicinformation

Exogenous variables:•Types of information•Fusion performed on theinformation, and qualityof this process•Training to use retrieveand present information •Whether the info was

retrieved in original form

•Whether the presentationof the info introduces errors

F(…): Correctness of information object• If info comes from an organic source, correctness =

organic correctness• If info is from network, correctness is the original

correctness “multiplied” by the probability the info wasretrieved and presented in original form.

• If info underwent fusion (esp. if taken from multiplesources), correctness is “multiplied” by additional factorrepresenting fusion effectiveness.

Computing Correctness for Individual Information: Detailed Function

!!!!General


OFTASDC3I

EBR




Quality ofOrganic

Information


Information

Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance




Exogenous variables:•Track, capability, intentinformation, all instandard formats•No fusion performed•Pilots will use organicinformation in preferenceto radio-reportedinformation•Pilots have sufficienttraining to use radio

•Assuming voice signalreaches blue AC with no infodegradation, but has only 70%chance of being audible

•Military vocoder repeatsauditory errors

F(…): Correctness of information object• Assumed correctness for each of blue AC’s own tracks• If info received from vocoder, approximately a 70%

chance that message will have been heard correctlyFusion does not apply in this case.

Total: 100% of organic info objects are correct; only 70% ofvoice-reported info objects are correct (others aregarbled to point of unusability)

Computing Correctness for Individual Information: Detailed Function, Voice Only

!!!!Voice


OFTASDC3I

EBR




Quality ofOrganic

Information


Information

Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance




Exogenous variables:•Track, capability, intentinformation, all instandard formats•Fusion consolidates bluetracks only•Pilots will use F-15 radarinformation in preferenceto AWACS information•Pilots have sufficienttraining to use Link 16display and radio

•Assuming Link 16 reachesblue AC with no infodegradation

•Display screen assumed tobe error free within screenresolution

F(…): Correctness of information object• Assumed correctness for each of blue AC’s own tracks• No errors introduced by Link 16 or info display (and

pilots adequately trained to use display)Total: 100% of info objects are correct, whether organic or

shared

Computing Correctness for Individual Information: Detailed Function, Link 16

!!!!Link 16


OFTASDC3I

EBR

Computing Extent of Shared Information:Detailed Function

f(…)

Whether senderand receiver arepart of the samecollaborativegroup


Ease of Use



Probability that senderwill attempt to shareinformation withreceivers

Number of communications“hops” between sender andreceiver

Relevance

Completeness

Timeliness

Accuracy

Currency

Consistency

Correctness


Precision

Quality

Objective Measures

Completeness

Timeliness

Accuracy

Currency

Consistency

Extent

Correctness


Precision

Quality

Objective Measures

Uncertainty

Extent: Proportion of force entities that share information



Organizational & Ind. Behavior


QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdepend

Syn

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Deg. Of Eng

Whethersender cancommunicateinfo withreceiver

Whetherinformation canbe physicallyshared acrossnetwork

Matrix showing probabilities that particular informationelements have been shared with particular users

For each element of the matrix, Pr(shared) is the product of:• the probability that the info is retrievable;• the probability the sender and receiver are part of the same

collaborative group;• the probability the sender and receiver can communicate

within the collaborative group;• the probability the sender attempts to share the information

with the receiver; and• the probability the information is not degraded as a function

of the number of “hops” between sender and receiver.

!!!!General


OFTASDC3I

EBR


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Innovativeness

Agility

Timeliness

Relevance

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

Adaptability

Objective Measures

Mode of D. M.

Precision

Currency

Consistency

Quality of Individual Sensemaking andDecision Making: Timeliness (Qualitative)




Ease of Use

f(…)

•Whether the focus ofinteractions is on informationgathering and validation orsensemaking/decision making•Whether all relevant membersare participating•Whether the intensity of theinteractions matches therequirements of the mission

•Whether the command structure allowsfor flexible roles and distributed decisionmaking

Exogenous variables:training, experience, etc.





QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdepend

Synchronization

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Engagement

Risk Propensity

F(…): Timeliness of Sensemaking / Decision Making• If information is shared among all participants, less time is spent gathering

and validating information, improving the timeliness of sensemaking anddecision making

• Real time interactions result in more efficient use of time, improving thetimeliness of sensemaking and decision making

• Flexible command structures allow force members to make decisions withfewer requirements, shortening decision-making times

!!!!General


OFTASDC3I

EBR


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Innovativeness

Agility

Timeliness

Relevance

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

Adaptability

Objective Measures

Mode of D. M.

Precision

Currency

Consistency

Quality of Individual Sensemaking andDecision Making: Timeliness

Voice Only (Qualitative)Degree of Information “Share-ability”



Ease of Use

f(…)

•Whether the focus ofinteractions is on informationgathering and validation orsensemaking/decision making•The number of participants•Whether the intensity of theinteractions matches therequirements of the mission







QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdepend

Synchronization

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Engagement

Risk Propensity

F(…): Timeliness of Sensemaking / Decision Making• Participants spend most of their time gathering and validating information

from AWACS and other blue AC radars• Voice communications adds delay over visual communications• Inflexible command structures require a variety of explicit checks and

permissions before engagement decisions can be made

!!!!General


OFTASDC3I

EBR


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Innovativeness

Agility

Timeliness

Relevance

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

Adaptability

Objective Measures

Mode of D. M.

Precision

Currency

Consistency


Link 16 (Qualitative)Degree of Information “Share-ability”



Ease of Use

f(…)








QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdepend

Synchronization

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Engagement

Risk Propensity

F(…): Timeliness of Sensemaking / Decision Making• Participants automatically receive all relevant information available from

AWACS and other blue AC radars, so pilots incur no delays bycommunicating this information verbally

• Near-real time visual information displays are much faster than voicetransmissions

• Flexible command structures allow pilots to engage targets, and supportengaging pilots, directly. (Commanders only intervene when necessary.)

!!!!General


OFTASDC3I

EBR

DEFINITIONS OF ATTRIBUTES


OFTASDC3I

EBR


Measures quality in reference to criteria that are determined by the situationFitness for UseMeasures

Extent to which information relevant to ground truth is collectedCompleteness

tuu


Proportion of information collected that is related to task at handRelevance

Appropriateness of precision of information for a particular useAccuracy

Extent to which currency of information is suitable to its useTimeliness

Level of measurement detail of information itemPrecision

Extent to which information is consistent with prior informationConsistency

Age of informationCurrency

Extent to which information is consistent with ground truthCorrectness

DefinitionAttribute

Information gathered by individual sensors that is not shared and is unavailable to the network

Metrics(Click Here)

Attribute Summary(Click Here)


OFTASDC3I

EBR


Degree to which presentation of information facilitates desired useEase of Use

Proportion of nodes that can retrieve various sets of information. Determinedby the following:•Awareness of Information: Degree to which the existence of the information isadvertised to force member

•Access to Information: Degree to which access to information is controlled

• Meta-data of Information: Degree to which information has labels describing what it isand how it may be used (facilitates indexing and searching)

Quantity ofRetrievableInformation

Extent to which collected information is postedQuantity of PostedInformation

DefinitionAttribute

The degree to which information could be shared among force entities

Metrics(Click Here)



OFTASDC3I

EBR



Extent to which information relevant to ground truth is obtainedCompleteness

tuu


Proportion of information retrieved that is related to task at handRelevance

Appropriateness of precision of information for a particular useAccuracy

Extent to which currency of information is suitable to its useTimeliness


Extent to which information is internally consistent with prior information/awareness / understanding

Consistency


Extent to which information is consistent with ground truthCorrectness

DefinitionAttribute

Information gathered by individuals from the network and organic sources

Metrics(Click Here)



OFTASDC3I

EBR

Quality of Individual Sensemaking

• Awareness- An individual’s holistic view of the battlespacethat includes mission constraints, environment, time spacerelationships, the capabilities and intentions of red, blue, andneutral forces and an assessment of the associated uncertainties

• Understanding- An individual’s recognition of patterns,cause and effect relationships, dynamic futures, andopportunities and risks


OFTASDC3I

EBR

Quality of Individual Sensemaking:Awareness

Subjective assessment of confidence in awarenessUncertainty


Extent to which relevant awareness is obtainedCompleteness

tuu


Proportion of awareness obtained that is related to task at handRelevance

Appropriateness of precision of awareness for a particular useAccuracy

Extent to which currency of awareness is suitable to its useTimeliness

Level of granularity of awarenessPrecision

Extent to which awareness is internally consistent with prior awarenessConsistency

Time lag of awarenessCurrency

Extent to which awareness is consistent with ground truthCorrectness

DefinitionAttribute Metrics(Click Here)



OFTASDC3I

EBR


Subjective assessment of confidence in understandingUncertainty


Extent to which relevant understanding is obtainedCompleteness


Proportion of understanding obtained by force member that is related to task at handRelevance

Appropriateness of precision of understanding for a particular useAccuracy

Extent to which currency of understanding is suitable to its useTimeliness

Level of granularity of understandingPrecision

Extent to which understanding is internally consistent with prior understandingConsistency

Time lag of understandingCurrency

Extent to which understanding is consistent with ground truthCorrectness

DefinitionAttribute Metrics(Click Here)



OFTASDC3I

EBR

Quality of Individual Decisions I


Type of decision making process utilized (naturalistic, dominated, min-max, expected utility)Mode of Decision Making

Extent to which decisions are consistent with existing understanding, command intent and valuesAppropriateness

Subjective assessment of confidence in decisionsUncertainty

Measures quality in reference to criteria that are determined by the situationFitness for Use Measures

Extent to which relevant decisions encompass the necessary:

•Depth: range of actions and contingencies included

•Breadth: range of force elements included

•Time: range of time horizons included

Completeness


Proportion of decisions that are significant to task at handRelevance

Appropriateness of precision of decisions for a particular useAccuracy

Extent to which currency of decision making is suitable to its useTimeliness

Level of granularity of decisionsPrecision

Extent to which decisions are internally consistent with prior understanding and decisionsConsistency

Time lag of decisionsCurrency

DefinitionAttribute

The extent to which an individual’s decisions build upon awareness and understanding



OFTASDC3I

EBR

Quality of Individual Decisions II

Degree to which decision facilitates force entities’ ability to alter the decision, decisionmaking participants and/or decision making process and implement appropriatemodifications

Adaptability

Agility

Degree to which decision reflects novel ways to perform known tasks and/or developsnew ways of doing novel tasks

Innovativeness

Degree to which decision allows force entities to maintain flexibility (i.e., incorporatesmultiple ways of succeeding)

Flexibility

Degree to which decision is relevant and timelyResponsiveness

Degree to which decision is dominant across a range of situations and degradationconditions

Robustness

DefinitionAttribute Attribute Summary(Click Here)


OFTASDC3I

EBR

Quality of Interactions:Definitions and Explanations

• Interactions involve force entities actively sharing information, anddeveloping awareness, understanding and/or making decisions (developingplans) in a collaborative fashion while working together toward a commonpurpose

• The focus of interactions: information sharing, developing and sharingawareness, developing and sharing understandings, making decisions

• Attributes of interactions– Depth, breadth, intensity, agility

• Contributing attributes– Individual Characteristics: risk propensity, competence, trust, organizational

identification, confidence– Organizational Characteristics: risk propensity, competence, trust, confidence,

size, hardness, diversity, permanence, autonomy, structure, interdependence– Organizational & Individual Behaviors: cooperation, efficiency

synchronization, engagement, team vs. task balance


OFTASDC3I

EBR

Degree of individual’s expectation that other members are reliableConfidence

Extent to which individual’s identities align with organizational identitiesOrganizationalIdentification

Level of knowledge, skills, abilities, and attitudes (KSAAs)Competence

Extent to which individual is willing to rely on other membersTrust


DefinitionsAttributes

Quality of InteractionsContributing Attributes: Individual Characteristics



OFTASDC3I

EBR


Degree to which team members have interacted in the past on the same taskHardness

Number of team members involvedSize

Extent to which members have expectations of the reliability of the organizationConfidence

Extent to which members are willing to rely on one anotherTrust

Distribution of members knowledge, skills, abilities and attitudes (KSAAs)Competence

Extent to which members depend on one another for resources (materials, KSAAs,etc.)

Interdependence

Distribution of peer and authority relationships

• Layers of authority

• Functional Differentiation

• Connectedness within and across layers

• Directness of connections

Structure

Expected duration of organizationPermanence

Extent to which organization is externally or self directedAutonomy

Degree to which team members are heterogeneous or homogeneous acrossexogenous variables: experience, age, gender, etc.

Diversity


Quality of InteractionsContributing Attributes: Organizational Characteristics



OFTASDC3I

EBR

Extent to which all members actively and continuously participateEngagement

Extent to which members utilize one another’s resources so as to minimize costsand maximize benefits

Efficiency

Extent to which organization is conflicted, deconflicted, or synergisticSynchronization

Extent to which efforts are directed to organizational issues vs. relating to theobjective

Team vs. TaskBalance

Extent to which member(s) are willing and able to work togetherCooperation


Quality of InteractionsContributing Attributes:

Organizational and Individual Behaviors



OFTASDC3I

EBR


Extent to which shared information is consistent with ground truthCorrectness

Extent to which shared information is consistent within and across CoIConsistency

Age of shared informationCurrency

Level of measurement detail of shared information itemPrecision


Extent to which shared information relevant to ground truth is obtainedCompleteness

Appropriateness of precision of shared information for a particular useAccuracy

Proportion of shared information retrieved that is related to task at handRelevance

Extent to which currency of shared information is suitable to its useTimeliness

Proportion of information in common across force entities, within and acrosscommunities of interest (CoI)

Proportion of force entities that share information item

Extent




OFTASDC3I

EBR

Degree of Shared Sensemaking:Shared Understanding

Extent to which shared understanding is consistent with ground truthCorrectness

Extent to which shared understanding is consistent within and across CoIConsistency

Time lag of shared understandingCurrency

Level of granularity of shared understandingPrecision


Extent to which relevant shared understanding is obtainedCompleteness

Appropriateness of precision of shared understanding for a particular useAccuracy

Proportion of shared understanding that is related to task at handRelevance

Extent to which currency of shared understanding is suitable to its useTimeliness

Subjective assessment of confidence in shared understandingUncertainty

Proportion of understanding in common across force entities, within and acrosscommunities of interest (CoI)

Proportion of force entities that share a given understanding

Extent




OFTASDC3I

EBR

DEFINITION OF METRICS


OFTASDC3I

EBR

Measures quality in reference to criteria that are determined by thesituation

Fitness for UseMeasures

Percentage of ground truth relevant and needed information collectedCompleteness

Measures quality in reference to criteria that are independent of thesituation

Objective Measures

Proportion of information collected that is related to task at handRelevance

Degree to which precision matches what is needed (0=no match, 10=high degree ofmatching between precision level needed and available)

Accuracy

Degree to which currency matches what is needed (0=no match, 10=high degree ofmatching between currency level needed and available)

Timeliness


Degree of ‘deviation’ from previous informationConsistency


Correspondence with ground truth-correlation coefficient (0= no correspondencewith ground truth, 1= full correspondence with ground truth). Data matrixcomprised of relevant information items estimates (for instance: detection, ID,velocity, location, heading, etc.)

Correctness

MetricsAttribute Definitions(Click Here)


Quality of Organic InformationInformation gathered by individual sensors

that is not shared and is unavailable to the network


OFTASDC3I

EBR


Degree to which information is easy to use (0=low degree of ease of use,10=high degree of ease of use)

Ease of Use

Percentage of nodes that can retrieve various sets of information.Quantity ofRetrievableInformation

Percent of collected information postedQuantity of PostedInformation

MetricsAttribute

The degree to which information could be shared among force entities

Definitions(Click Here)



OFTASDC3I

EBR


Percentage of ground truth relevant and needed informationCompleteness


Proportion of information that is related to task at handRelevance

Degree to which precision matches what is needed (0=no match, 10=high degree ofmatching between precision)

Accuracy


Timeliness


Degree of ‘deviation’ from previous informationConsistency


Correspondence with ground truth-correlation coefficient (0= no correspondence withground truth, 1= full correspondence with ground truth). Data matrix comprised ofrelevant information items estimates (for instance: detection, ID, velocity, location, etc.)

Correctness

MetricsAttributeDefinitions(Click Here)


Quality of Individual InformationInformation gathered by individuals from the network and organic sources


OFTASDC3I

EBR

Quality of Individual Sensemaking:Awareness

Confidence level (0% =uncertain, 100%= certain) or confidence interval (95%, 90%, etc.)of awareness

Uncertainty


Percentage of ground truth picture included in awarenessCompleteness


Proportion of awareness that is related to task at handRelevance


Accuracy


Timeliness

Level of granularity of awarenessPrecision

Degree of ‘deviation’ from awareness gained from previous time periodConsistency

Time lag of awarenessCurrency

Correspondence with ground truth-correlation coefficient (0= no convergence, 1=fullconvergence between individual’s awareness and ground truth)

Correctness




OFTASDC3I

EBR

Quality of Individual Sensemaking:Understanding

Confidence level (0% =uncertain, 100%= certain) or confidence interval (95%, 90%,etc.) of awareness

Uncertainty


Percentage of ground truth picture included in understandingCompleteness


Proportion of understanding that is related to task at handRelevance


Accuracy


Timeliness

Level of granularity of understandingPrecision

Degree of ‘deviation’ from understanding gained from previous time periodConsistency

Time lag of understandingCurrency

Correspondence with ground truth-correlation coefficient (0= no convergence, 1=fullconvergence between individual’s understanding and ground truth)

Correctness




OFTASDC3I

EBR

NCWFocusedResearch

NCWKnowledge

Base

NCWFocused

Experiments

NCW Theory(Hypotheses)

ConceptualFramework

Metrics

Board of Directors

The NCW Framework InitiativeKey To Developing and Applying NCW Theory Across DoD Enterprise

Awareness

Education

Consulting

EnterpriseApplications

Relationships

Code of Best Practice

Tools

Methodology

•Analysis•Experiments•Exercises•Case Studies•T & E•Etc.


OFTASDC3I

EBR

Status of Framework Development

• “Getting NCW Theory and Metrics Right…”– Where we are

• Second generation framework– Being evolved through peer review

• Initial case study in progress (Air-to-Air combat)

– Where we are going• Additional peer review• Continual revision

• “…And Applied Enterprise-Wide”– Where we are

• Establishing collaborative partnerships

– Where we are going• Dissemination and education

– Symposium, workshop, web, brochure, tutorials

• Additional case studies– Sponsored, supported, encouraged


OFTASDC3I

EBR

Application of the NCW Framework to Air-to-Air Combat

• Objectives– Gain insight into how NCW is a new source of power in

Air-to-Air combat– Illuminate contribution of enabling capabilities in the NCW

value chain– Identify areas where data is needed– Assess utility of framework and identify needed

improvements• Approach

– Start with data from the JTIDS Operational Special Project– Apply NCW Framework to instantiate influence model– Capitalize on additional data and impute missing data– Identify sources of improved combat power


OFTASDC3I

EBR

Numerical Metrics for Case StudyCalculated with Analytica


OFTASDC3I

EBR














Force C2 EffectorsValue Added

Services


Quality of

Inter-actions

Information Sources


OFTASDC3I

EBR

Quality of Organic Information:Blue Tracks

Blue11, 12

Blue13, 14

0

0.25

0.5

0.75

1

BBBB QQQQ 4321

Q1B: Completeness: Detection

Q2B: Correctness: ID

Q3B: Correctness: Location

Q4B: Correctness: Velocity

0

0.25

0.5

0.75

1

BBBB QQQQ 4321

00.250.5

0.751

BBBB QQQQ 4321


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach


Information. Sources Value Added Services C2 Effectors

Force



Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance




Qualityof

Interactions

Influence of Networking onInformation “Share-ability”


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach


Information. Sources Value Added Services C2 Effectors

Force

Ease of Use


Computing Quantity of Posted Info

Exogenous variables:•Number of nodes•Number and size of files•Variables that impact howquickly nodes can transmit(CONOPS, coding schemes,etc.)•Policies determiningpriority for posting•Expiration age for eachtype of info objects•Maximum queue lengths

Whether nodes can transmit tonetwork

Posting channel numbers, types,and bandwidth (for data linksonly)

Prob of correct transmission

Adjustments to probability thatQoS will be delivered

In this scenario, quantity of posted info.equals quantity of retrievable info,except for probability of hearing voice

f(…)

Voice Only Link 16


OFTASDC3I

EBR

Quantity of Retrieved InformationPercent of Organic Information Retrieved

1Q

2Q3Q

4QCompleteness: Detection

Correctness: ID



00.250.5

0.751

BBBB QQQQ 4321

BBBB QQQQ 4321

BBBB QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321

00.250.5

0.751

00.250.5

0.751

Threat Tracks Blue Tracks

AWACS

Blue 11 & 12

Blue 13 & 14

Link 16 + Voice

BBBB QQQQ 4321

BBBB QQQQ 4321

BBBB QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321


Voice Only

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751


OFTASDC3I

EBR

Comparing MCPs Using Summary MetricsDegree of Networking and Quantity of Information Retrieved


0.28



Degree ofNetworking


“Share-ability”

Quality ofIndividualDecisions

Kill Ratio(Synchronization / Effectiveness)



Sensemaking

**


0.51.00.08

1.0


OFTASDC3I

EBR




Node Assurance



Connectivity

Capacity

Network Agility

Network Assurance

Quality of Service

Reach

Degree of NetworkingQuality of Organic Information


Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance




Qualityof

Interactions

Influence of Organic Info Quality and Degree of Info Sharing on



OFTASDC3I

EBR






Ease of Use

Consistency

TimelinessPrecision

TimelinessPrecision

RelevanceCurrency

AccuracyConsistency


Currency

Accuracy


Relevance



Quality of Individual Information:Correctness

•Correctness of organicinformation

Exogenous variables:•Fusion•Training •Whether the info was

retrieved in original form

•Whether the presentationof the info introduces errors

f(…)

Voice Only Link 16


OFTASDC3I

EBR

Quality of Individual Information:Voice Only vs. Link 16

1Q

2Q3Q


Correctness: ID



00.250.5

0.751

00.250.5

0.751

BBBB QQQQ 4321

BBBB QQQQ 4321

BBBB QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751


AWACS

Blue 11 & 12

Blue 13 & 14

Link 16 + Voice

BBBB QQQQ 4321

BBBB QQQQ 4321

BBBB QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321

TTTT QQQQ 4321


Voice Only

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751

00.250.5

0.751


OFTASDC3I

EBR

Comparing MCPs Using Summary MetricsQuality of Individual Information


0.28



Degree ofNetworking


“Share-ability”





Sensemaking

**


0.51.00.08

1.0

0.4

0.91


OFTASDC3I

EBR


Consistency

TimelinessPrecision

Currency

Accuracy


Relevance

Objective MeasuresFitness for Use


Currency

Consistency

Correctness

Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Precision


Influence of Information “Share-ability” and Nature andQuality of Interactions on Degree of Shared Information

Relevance

Completeness

Timeliness

Accuracy

Currency

Consistency

Extent

Correctness


Precision

Quality

Objective Measures

Completeness

Timeliness

Accuracy

Currency

Consistency

Extent

Correctness


Precision

Quality

Objective Measures

Uncertainty

Degree of Information “Share-ability”Quantity of Retrievable InfoQuantity of Posted InfoEase of Use





QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdependence

Synchronization

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchroniety

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Deg. Of Eng


OFTASDC3I

EBR

Computing Extent of Shared Information

Whether senderand receiver arepart of the samecollaborativegroup


Ease of Use



Probability that senderwill attempt to shareinformation withreceivers

Number of communications“hops” between sender andreceiver

Relevance

Completeness

Timeliness

Accuracy

Currency

Consistency

Correctness


Precision

Quality

Objective Measures

Completeness

Timeliness

Accuracy

Currency

Consistency

Extent

Correctness


Precision

Quality

Objective Measures

Uncertainty

Extent: Proportion of force entities that share information



Organizational &Individual Behavior


QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdepend

Syn

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Deg. Of Eng

Whethersender cancommunicateinfo withreceiver

Whetherinformation canbe physicallyshared acrossnetwork

f(…)


OFTASDC3I

EBR

Degree of Shared Information:Extent of Shared Track Information

0

0.25

0.5

0.75

1

Voice + Link 16

TTTT QQQQ 4321

Threat Tracks

Blue Tracks

Voice Only

1Q

2Q3Q


Correctness: ID



TTTT QQQQ 4321 0

0.25

0.5

0.75

1

0

0.25

0.5

0.75

1

TTTT QQQQ 4321

Threat Tracks

Blue Tracks

TTTT QQQQ 4321 0

0.25

0.5

0.75

1

All blue AC havethe same sharedinformationin this example(all listen to thesame voicechannel or receivethe same Link 16broadcasts)


OFTASDC3I

EBR

Comparing MCPs Using Summary MetricsDegree of Shared Information


0.28



Degree ofNetworking


“Share-ability”

Quality ofIndividualDecisions

Kill Ratio(Synchronization / Effectiveness)



Sensemaking

**


0.51.00.08

1.0

0.4

0.91

1.0

0.22


OFTASDC3I

EBR


Currency

Consistency

Correctness

Precision Uncertainty

Timeliness

Relevance

Accuracy

Completeness


Completeness

Timeliness

Accuracy

Extent


Currency

Consistency

Correctness

Precision

Quality

Objective Measures

Uncertainty


Currency

Consistency

Correctness

Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Precision


Completeness

Timeliness

Accuracy

Extent

Degree of Shared Sensemaking: Shared Understanding

Currency

Consistency

Correctness

Precision

Quality

Objective Measures

Uncertainty

Quality of Collaborative Decisions

Fitness for Use

Innovativeness

AgilityObjective Measures

Mode of D. M.

Precision

Currency

Consistency

Extent

Timeliness

Relevance

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

Adaptability


Timeliness

Relevance

Accuracy

Completeness

Appropriateness

UncertaintyInnovativeness

Agility

Responsiveness

Flexibility

Robustness

Adaptability

Objective Measures

Mode of D. M.

Precision

Currency

Consistency





QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdepend

Syn

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Distance

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Deg. Of Eng


Q of Posted Info Q of Retrievable Info Ease of Use

Risk Propensity

Influence of Information “Share-ability” andNature and Quality of Interactions on

Individual Awareness, Understanding, and Decisions: Timeliness


OFTASDC3I

EBR


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Uncertainty

Timeliness

Relevance

Accuracy

Completeness

Fitness for Use

Currency

Consistency

Correctness

Precision

Objective Measures


Innovativeness

Agility

Timeliness

Relevance

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

Adaptability

Objective Measures

Mode of D. M.

Precision

Currency

Consistency





Ease of Use

f(…)








QualityDepth

Breadth

Agility

Intensity

Response

Trust

Interdepend

Synchronization

Adaptability

Flexible

Selectivity

Mode

Latency

Confidence

Competence

Competence

Confidence

Hardness

Permanence

Structure

Efficiency

T vs. T

Quantity

Robustness

Innovative

Reach

Continuity

Synchronicity

Risk Prop

Org. Ident

Risk Prop

Diversity

Trust

Size

Autonomy

Cooperation

Engagement

Risk Propensity

f(…)

Voice Only Link 16


OFTASDC3I

EBR

Comparing MCPs Using Summary MetricsQuality of Individual Sensemaking and Decisions (Notional)


0.28



Degree ofNetworking


“Share-ability”





Sensemaking

**


0.51.00.08

1.0

0.4

0.91

1.0

0.22

0.450.91

0.45

0.91


OFTASDC3I

EBR

Qualityof Interactions

Quality of Collaborative Decisions


Mode of D. M.

Timeliness

Precision

Innovativeness

Agility

Relevance

Currency

Consistency

Extent

Accuracy

Completeness

Appropriateness

Uncertainty

Responsiveness

Flexibility

Robustness

Adaptability

Quality of Individual Decisions

Precision

Currency

Consistency

Uncertainty

Relevance

Accuracy

Completeness

Appropriateness

Mode of D. M.


Timeliness



Synchronized Decisions/Plans


Synchronized Actions Synchronized Entities


Achievement of Objectives Agility Timeliness Efficiency

Influence of Individual and Collaborative Decisions onSynchronization and Effectiveness / Agility


OFTASDC3I

EBR

Major Products








Force







Information Sources C2 Effectors

Value Added

Services


Physical Domain

Social Domain

Information Domain

Cognitive Domain

Qualityof

Inter-actions

Air-to-Air Case Study




Degree of Information

“Share-ability”

Quality of Individual DecisionsKill Ratio


Quality of Organic Info

Quality of Individual

Sensemaking1.0

0.28

0.51.00.08

1.0

0.4

0.91

3.10:1

8.11:1

1.0

0.22

0.450.91

0.45

0.91


OFTASDC3I

EBR

Accomplishments

“Getting NCW Theory and Metrics Right…”

– Extension and refinement of framework• Sensemaking

• Social domain

• Quantitative metrics

• Operational application

“…And Applied Enterprise-Wide”- Initial application of framework

• Ongoing partnerships with allies (UK, Australia, Canada)• Joint Force C2 concept (JCS)• DPG Study 9: Alternative Interoperability Strategies (JCS/J8 led)• Multinational LOE (JFCOM/J7, J9 led)• Transformation of GCCS (DISA)


OFTASDC3I

EBR

Degree of Networking:Net Ready Nodes

Degree to which node has the ability to post and retrieve information indesired formats and places

Posting andRetrieving CapabilitySupport

Number and types of access modes supportedConnectivity

Number and types of collaboration applications supportedCollaborationSupport

Ability of the node to exchange and use data (incorporates data exchange rate,CPU, memory, disk storage, etc.)

Capacity

Extent to which node supports services that facilitate the assurance ofinformation in the areas of privacy, availability, integrity, authenticity, andnonrepudiation

Node Assurance

DefinitionAttribute

Nodes that are capable of sharing information andcollaborating with others


OFTASDC3I

EBR

Percentage of nodes that can post and retrieve in desired formatsPosting andRetrievingCapability Support

Vector of number and types of access modes supportedConnectivity

Number and types of collaboration applications supportedCollaborationSupport

Largest bandwidth the node can access (56K bps, 1.5Mbps, etc.)Capacity

Categorical rating from “highly secure” to “not secure”

(estimated from assessment of node’s installed security software, hardware,and usage policies)

Node Assurance

MetricsAttribute

Degree of Networking:Net Ready Nodes

Nodes that are capable of sharing information andcollaborating with others

network centric

Documents

ncw theory

force ncw measures

ncw help

ncw conceptualframework

ncw framework elements

networkcentric force

new conceptual framework

new theory