implementing critiquing question #3 using patterns

Implementing Critiquing Question #3 using Patterns

UT RKF Group

June 26, 2002

The process of authoring a pattern is no different from entering an ordinary CMAP. The SME would start off by selecting “Enter Knowledge” from SHAKEN’s main menu followed by selecting “Create a New Concept” in the subsequent menu.

The pattern would then be given a name.

The concept “Pattern” would then be selected as the superclass.

The pattern would be drawn like any other CMAP. The only thing special about entering a pattern is the following:

• The SME specifies a critique criterion using property values.

• The SME associates a question with a pattern.

Let’s see how this is done.

Here’s one way that a question might be associated with a pattern.

An new option called “Associate Question” is added in a pop up menu. The SME would access this option by right clicking on the screen and selecting it.

This will bring up a window in which the SME enters the question.

A critique criterion is added to a pattern via the existing property values mechanism.

Another piece of information that can be added to a pattern is a text-generation template. We envision this being used by SHAKEN in generating the final report.

The next couple of slides is a proposal for this mechanism.

The SME would enter a description of the pattern.

The SME can associate a phrase with a node in the pattern by highlighting the phrase and then pressing the select button.

Continuing with the example, the SME selects “aircraft” from the text template.

This will bring up the pattern, and the SME can select the node by clicking on it.

Now the SME selects the node labeled “Aviation-Batallion” from the Pattern

Selected node.

The label on the selected node (put in angle brackets) replaces the selected text in the template. The SME repeats this process for other words/phrases in the template, then selects “Done”.

When SHAKEN generates a final report critiquing a COA, the phrase <aircraft> will be replaced by the called tag of the COA node that matches the Pattern node.

Implementation of the SME’s text template.

• The output of the SME’s text template can be represented in SHAKEN’s .km file as:

(_Destroy-Artillery-Early1363 has

(instance-of (Destroy-Artillery-Early))

(critique-score (_Critique-Score-Value1377))

(coa-description ('("The" _Aviation-Batallion1365 "destroy" _Artillery-Battery1366 "before" _Attack1369)))

(has-pattern (_Destroy1376))

(new-objects (_Destroy1376 _Aviation-Batallion1365 _Artillery-Battery1366 _Time-Interval1367

_Time-Interval1368 _Attack1369 _View1370 _Allegiance-Value1371 _Property-Node1372

_Allegiance-Value1373 _Property-Node1374 _Critique-Score-Value1377 _Property-Node1375)))

*With the exception of part circled in red, the above contents can be found in SHAKEN in the file Destroy-Artillery-Early.km under the user porter.

The output of the SME’s text template.

To critique a COA, the SME would select “Ask a Question” from SHAKEN’s main menu. They would then enter the name of the COA to be critiqued.

This will bring up SHAKEN’s Q/A subsystem.

Assume the SME asks the first question in the list. SHAKEN then applies the pattern associated with that question. At this point, SHAKEN would call UT’s pattern-match function (seehttp://www.cs.utexas.edu/users/mfkb/RKF/API.txt for the APIs of UT’s functions).

The output from the function will then be used to construct the critique report shown later. Let’s first look at the output generated by UT’s pattern-match function.

The question associated with the pattern Destroy-Artillery-Early.

http://www.cs.utexas.edu/users/mfkb/RKF/API.txt

To clarify the output, here’s a graphical view of it. We’re not suggesting that this be generated automatically.

Application of the heuristic rewrite: if Time-Interval time-of Event1 and Event1 causes Event2 then Time-Interval time-of Event2

Application of the heuristic rewrite: if Entity agent-of Event1 and Event1 causes Event2 then Entity agent-of Event2

*The other parts of the pattern and COA was able to be matched without the use of any rewrites. These

correspondences are not shown here for the sake of brevity

This critique is generated by instantiating the text template:“The <aircraft> destroy <enemy artillery> before <attack>.”with the called tags of the nodes that match selected nodes

of the Pattern. In this example, the underlined phrases come from called tags on nodes in COA-J.

Options for producing an explanation are shown on the next slide.

SRI computes the overall score by (somehow)

tallying the individual scores provided by the

critiquers.

Explaining a Critique• low road: show the COA filtered by the pattern that

generated the critique. Invite the SME to see for himself how the pattern and the COA got aligned by the matcher. This requires no new functionality (beyond what we’ve already proposed) because SHAKEN can get the information needed to filter the COA from the output our pattern-match function.

• high road: show the COA and the pattern and the correspondence between the two. Showing this information graphically would be best. This information is returned by the pattern matcher, but presenting it to the SME graphically would require new functionality.

implementing critiquing question #3 using patterns

Documents