Ontology Aware Software Service Agents: Meeting Ordinary User

Needs on the Semantic Web

Muhammed Al-Muhammed

April 19, 2005

April 19, 2005 (2)

The Challenge

Reduce information overload Find and use services

I want to see a dermatologist next week; any day would be ok for me, at 4:00 p.m. The dermatologist must be within 20 miles from my home and must accept my insurance.

April 19, 2005 (3)

Thesis Statement

Hypothesis: it is possible to automate everyday tasks whose invocation results in establishing agreed-upon relationships in a domain ontology

Validation: proof-of-concept prototype

April 19, 2005 (4)

Approach

Task ontology Domain ontology Process ontology

Characteristics Task specification: Free-form text Request recognition: find best task ontology Task execution

• Specialize task ontology processes

• Execute generated code

April 19, 2005 (5)

Domain Ontology

Components Object sets = concepts Relationship sets Constraints

Uses Task knowledge Task recognition

April 19, 2005 (6)

Domain Ontology

Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->

April 19, 2005 (7)

Domain Ontology

Augmented with data frames A data frame defines information about a

concept Its internal and external representation Its contextual keywords or phrases Operations along with contextual keywords or phrases

April 19, 2005 (8)

Data FramesTime

…

textual representation: “([2-9]|1[012]?)\s* :\s*([0-5]\d)\s*[AaPp]\s* \.?\s* [Mm]\s* \.?)”

…

end

Distance

internal representation: real

textual representation: ((\d+(\.\d+)?)|(\.\d+))

context keywords/phrases: miles | mile | kilometers | …

Within(d1: Distance, d2: Distance)

returns (Boolean)

contextual keywords/phrases: less than | | …

…

end

April 19, 2005 (9)

Task Recognition

A task domain determination Input: a task specification, domain ontologies Output: a marked domain ontology A domain-independent process

April 19, 2005 (10)

Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->

Appointment …

context keywords/phrase: “appointment |want to see a |…”

Dermatologist …

context keywords/phrases: “([D|d]ermatologist) | …”

I want to see a dermatologist next week; any day would

be ok for me, at 4:00 p.m. The dermatologist must be

within 20 miles from my home and must accept my

insurance.

April 19, 2005 (11)

Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->

Appointment …

context keywords/phrase: “appointment |want to see a |…”

Dermatologist …

context keywords/phrases: “([D|d]ermatologist) | …”

I want to see a dermatologist next week; any day would

be ok for me, at 4:00 p.m. The dermatologist must be

within 20 miles from my home and must accept my

insurance.

April 19, 2005 (12)

Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->

Appointment …

context keywords/phrase: “appointment |want to see a |…”

Dermatologist …

context keywords/phrases: “([D|d]ermatologist) | …”

I want to see a dermatologist next week; any day would

be ok for me, at 4:00 p.m. The dermatologist must be

within 20 miles from my home and must accept my

insurance.

April 19, 2005 (13)

Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->

Appointment …

context keywords/phrase: “appointment |want to see a |…”

Dermatologist …

context keywords/phrases: “([D|d]ermatologist) | …”

I want to see a dermatologist next week; any day would

be ok for me, at 4:00 p.m. The dermatologist must be

within 20 miles from my home and must accept my

insurance.

April 19, 2005 (14)

Appointment …

context keywords/phrase: “appointment |want to see a |…”

Dermatologist …

context keywords/phrases: “([D|d]ermatologist) | …”

I want to see a dermatologist next week; any day would

be ok for me, at 4:00 p.m. The dermatologist must be

within 20 miles from my home and must accept my

insurance.

Date …NextWeek(d1: Date, d2: Date)returns (Boolean)context keywords/phrases: next week | week from now | …

Distanceinternal representation : realtextual representation: ((\d+(\.\d+)?)|(\.\d+))context keywords/phrases: miles | mile | kilometers | …Within(d1: Distance, “20”)returns (Boolean)context keywords/phrases: within | not more than | | …return (d1d2)…end

April 19, 2005 (15)

Process Ontology

Process to execute tasks in a domain Statenet

States Transitions, based on ECA rules

April 19, 2005 (16)

Process Ontology@create

initialize

ready

missing information

task-view = get-from-system(task-view)if (still missing values) task-view = get-from-user(task-view)

no missing information

task-view != null

schedule-appointment(task-view.Person.Name,task-view.Service Provider.Name, task-view.Date,task-view.Time, task-view.Address)report that the appointment is scheduled

task-view = null

report that the appointment cannotbe scheduled

ready to schedule

initial task-view ready

@process ontology(domain ontology)

task-view = create-task-view(domain ontology)task-constraints = create-task-constraints(task-view)

@create

initialize

ready

missing information

task-view = get-from-system(task-view)if (still missing values) task-view = get-from-user(task-view)

no missing information

task-view != null

schedule-appointment(task-view.Person.Name,task-view.Service Provider.Name, task-view.Date,task-view.Time, task-view.Address)report that the appointment is scheduled

task-view = null

report that the appointment cannotbe scheduled

ready to schedule

initial task-view ready

@process ontology(domain ontology)

task-view = create-task-view(domain ontology)task-constraints = create-task-constraints(task-view)

April 19, 2005 (17)

Task Execution

Domain-independent subprocesses Coded once Specialized for a domain

A domain-dependent subprocess Fully determined (given the task specification and

domain ontology) Automatically generated

April 19, 2005 (18)

Task View Creation

Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->Appointment

Place

Insurance

Service Provider

Person

NameDoctor

Pediatrcian

Service Description

Duration

Medical Service Provider

Auto Service Provider Auto Mechanic

Dermatologist

Address

Cost

Date

Time

has

is at

is on

has

provides

has

accepts

hashas

"IHC"

is with

is for

is at

is at

has

"DMBA"

is at

->

April 19, 2005 (19)

Task View Creation

Dermatologist Insurance

Appointment

Address

Person

Name

Date

Time

is at

is on

has

is with

is for

is at

is at

has

is at

->

acceptsDermatologist Insurance

Appointment

Address

Person

Name

Date

Time

is at

is on

has

is with

is for

is at

is at

has

is at

->

accepts

April 19, 2005 (20)

Creation of Additional Task Constraints

Date …NextWeek(d1: Date, d2: Date)returns (Boolean)context keywords/phrases: next week | week from now | ……endDistanceinternal representation: realtextual representation: ((\d+(\.\d+)?)|(\.\d+)) context keywords/phrases: miles | mile | kilometers | … Within(d1: Distance, “20”)returns (Boolean)context keywords/phrases: within | not more than | | …return (d1d2)…end

Dermatologist Insurance

Appointment

Address

Person

Name

Date

Time

is at

is on

has

is with

is for

is at

is at

has

is at

->

acceptsDermatologist Insurance

Appointment

Address

Person

Name

Date

Time

is at

is on

has

is with

is for

is at

is at

has

is at

->

accepts

April 19, 2005 (21)

Creation of Additional Task Constraints

Taskimposed constraints:

NextWeek(d1: Date, d2: Date)

Person(x) is at Address(a1) and Dermatologist(y) is at Address(a2) and

Within(DistanceBetween(a1, a2), “20”)

i2 (Dermatologist(y) accepts Insurance(i2 ) and Equal(i1, i2))

April 19, 2005 (22)

Obtaining Information from the System

Appointment ->

Dermatologist

Insurance

Time “4:00”

Date

Person

Address

Name

Taskimposed constraints:

NextWeek(d1: Date, d2: Date)

Person(x) is at Address(a1) and Dermatologist(y) is at Address(a2) and

Within(DistanceBetween(a1, a2), “20”)

i2 (Dermatologist(y) accepts Insurance(i2 ) and Equal(i1, i2))

April 19, 2005 (23)

Obtaining Information from the System

Appointment ->

Dermatologist Dermatologist0 Dermatologist1

Insurance “IHC” “DMBA”

Time “4:00”

Date “5 Jan 05” “6 Jan 05”

Person

Address “Orem 600 State St.” “Lindon 12 Main St.”

Name “Dr. Carter” “Dr. Larry”

Taskimposed constraints:

NextWeek(d1: Date, d2: Date)

Person(x) is at Address(a1) and Dermatologist(y) is at Address(a2) and

Within(DistanceBetween(a1, a2), “20”)

i2 (Dermatologist(y) accepts Insurance(i2 ) and Equal(i1, i2))

April 19, 2005 (24)

Obtaining Information from the User

Appointment ->

Dermatologist Dermatologist0 Dermatologist1

Insurance “IHC” “DMBA”

Time “4:00”

Date “5 Jan 05” “6 Jan 05”

Person

Address “Orem 600 State St.” “Lindon 12 Main St.”

Name “Dr. Carter” “Dr. Larry”

Taskimposed constraints:

NextWeek(d1: Date, d2: Date)

Person(x) is at Address(a1) and Dermatologist(y) is at Address(a2) and

Within(DistanceBetween(a1, a2), “20”)

i2 (Dermatologist(y) accepts Insurance(i2 ) and Equal(i1, i2))

April 19, 2005 (25)

Obtaining Information from the User

Appointment ->

Dermatologist Dermatologist0 Dermatologist1

Insurance “IHC” “DMBA”

Time “4:00”

Date “5 Jan 05” “6 Jan 05”

Person Person100

Address “Orem 600 State St.” “Lindon 12 Main St.” “Provo 300 State St.”

Name “Dr. Carter” “Dr. Larry” “Lynn Jones”

Taskimposed constraints:

NextWeek(d1: Date, d2: Date)

Person(x) is at Address(a1) and Dermatologist(y) is at Address(a2) and

Within(DistanceBetween(a1, a2), “20”)

i2 (Dermatologist(y) accepts Insurance(i2 ) and Equal(i1, i2))

April 19, 2005 (26)

Constraint Satisfaction

Appointment ->

Dermatologist Dermatologist0 Dermatologist1

Insurance “IHC” “DMBA”

Time “4:00”

Date “5 Jan 05” “6 Jan 05”

Person Person100

Address “Orem 600 State St.” “Lindon 12 Main St.” “Provo 300 State St.”

Name “Dr. Carter” “Dr. Larry” “Lynn Jones”

Taskimposed constraints:

NextWeek(d1: Date, d2: Date)

Person(x) is at Address(a1) and Dermatologist(y) is at Address(a2) and

Within(DistanceBetween(a1, a2), “20”)

i2 (Dermatologist(y) accepts Insurance(i2 ) and Equal(i1, i2))

April 19, 2005 (27)

Constraint Satisfaction

Person(Person100) is at Address(“Provo 300 State St.”) and Dermatologist(Dermatologist0) is at Address(“Orem 600 State St.”) and Within(DistanceBetween(“Provo 300 State St.”, “Orem 600 State St.”), “20”) (Dermatologist(Dermatologist0) accepts Insurance(“IHC” ) and Equal(“IHC”,“IHC”) or Dermatologist(Dermatologist0) accepts Insurance(“DMBA”) and Equal(“IHC”,“DMBA”))

Person(Person100) is at Address(“Provo 300 State St.”) and Dermatologist(Dermatologist1) is at Address(“Lindon 12 Main St.”) and Within(DistanceBetween(“Provo 300 State St.”, “Lindon 12 Main St.”), “20”) (Dermatologist(Dermatologist1) accepts Insurance(“DMBA”) and Equal(“IHC”,“DMBA”))

April 19, 2005 (28)

Constraint Satisfaction

Task imposed constraints: Person(Person100) is at Address((“Provo 300 State St.”) and Dermatologist(Dermatologist0) is at Address(“Orem 600 State St.”) and Within(DistanceBetween(“Provo 300 State St.”, “Orem 600 State St.”), “22”)

Appointment ->

Dermatologist Dermatologist0

Insurance “IHC” “DMBA”

Time “4:00”

Date “5 Jan 05”

Person Person100

Address “Orem 600 State St.” “Provo 300 State St.”

Name “Dr. Carter” “Lynn Jones”

April 19, 2005 (29)

Negotiation

Task imposed constraints: Person(Person100) is at Address((“Provo 300 State St.”) and Dermatologist(Dermatologist0) is at Address(“Orem 600 State St.”) and Within(DistanceBetween(“Provo 300 State St.”, “Orem 600 State St.”), “22”)

Appointment ->

Dermatologist Dermatologist0

Insurance “IHC” “DMBA”

Time “4:00”

Date “5 Jan 05”

Person Person100

Address “Orem 600 State St.” “Provo 300 State St.”

Name “Dr. Carter” “Lynn Jones”

April 19, 2005 (30)

Finalization

NextWeek(“28 Dec 04”, “5 Jan 05”)Person(Person100) is at Address(“Provo 300 State St.”) and Dermatologist(Dermatologist0) is at Address(“Orem 600 State St.”) and Within(DistanceBetween(“Provo 300 State St.”, “Orem 600 State St.”), “22”) i2 (Dermatologist(Dermatologist0) accepts Insurance(i2) and Equal(“IHC”, i2))

Appointment 7

Dermatologist0

“IHC” “DMBA”

“4:00”

“5 Jan 05”

Person100

“Orem 600 State St.” “Provo 300 State St.”

“Dr. Carter” “Lynn Jones”

April 19, 2005 (31)

Testing

“Black box” testing Concept recognition Constraint recognition Domain recognition Execution performance

“White box” testing for the processes of the system such as Negotiation Obtaining information from the user

April 19, 2005 (32)

Delimitations

Recognition and execution of complex tasks Compositional tasks Alternative tasks Conditional tasks

Graphical or vocal specification of tasks

April 19, 2005 (33)

Contributions

Simplification of everyday task execution—find and use versus specify

Automatic process generation for task execution Domain recognition and specialization Automatic information gathering from both system and user Constraint satisfaction Negotiation

Prototype implementation