seminar on new trends in intelligent systems and soft computing october 2-3.2003, granada, spain 1...

Seminar on New Trends in Intelligent Systems and Soft Computing October 2-3.2003, Granada, Spain

1

New Trends in Intelligent Systems and Soft Computing

Towards an Increased Role of Natural Language

Janusz Kacprzyk

Systems Research Institute,Polish Academy of SciencesUl. Newelska 6

01-447 Warsaw, PolandEmail: [email protected]


2

Contents

What is intelligence, from the perspective of psychology and cognitive science, and of information technology,

What are „intelligent systems”, Why is soft computing important, Why is the use of (quasi)natural language so crucial

in intelligent systems, Why is Zadeh’s computing with words and

perceptions paradigm viable, intuitive and constructive,

Some implementation (linguistic database summaries)


3

What is intelligence?

Initially: psychology, cognitive science

Different views (an exact definition of intelligence is probably impossible), for instance:

an ability to handle complexity and solve problems in some useful context as, e.g., reaching an agreement, finding a solution to the quadratic equation,

an ability to protect the organism from bodily risks and to satisfy its wants with the least possible chance of failure,...


4

Nature of intelligence

Two basic schools of thought on the nature of intelligence:

One general intelligence (Eysenck, Galton, Jensen, Spearman, ...)

Multiple intelligences (Gardner, Sternberg, Thurstone,...)


5

One general intelligence

For instance, Eysenck (1982):

There is one general factor governing the level of intelligence of an individual

„Proof”: a high positive correlation (positive manifold) between

tests of cognitive abilities (Spearman, 1904), e.g., good verbal abilities are usually linked to mathematical abilities,

A high correlation of reaction time and IQ, ...


6

Multiple intelligence

More than a single type of intelligence, but how many?

For instance:

Gardner (1983): 7 different forms of intelligence:

Linguistic, musical, spatial, bodily, interpersonal, intrapersonal, and logico-mathematical

Solid biological basis (seven different areas of brain)! Brain as a major determinant of intelligence!


7

Multiple intelligence, continued

Sternberg (1985): Social and contextual factors apart from human abilities!

Two different types of intelligence:

Analytic (academic), rather formulated by others and well defined, with all information needed for solution, have a single answer, ...

Practical: problems poorly defined, require problem recognition and formulation, have various acceptable solutions, require experience, motivation, etc.

„Practical”: very relevant for us!


8

Multiple intelligence, continued

Thurstone (1938) – 13 different factors:

Spatial, perceptual, numerical, verbal, memory, arithmetic reasoning, deductive abilities

Also (e.g. Thurstone, 1938; Guildford, 1967):

Intelligence is composed of 4 contents, 5 operations and 6 processes (= 120 combinations of abilities)

Many aspects very relevant to „intelligent systems”!!!


9

Intelligence in a „machine context”

Turing’s (1950) test:We have a person, a machine, and an interrogator, in different rooms. The interrogator is to determine which of the other two is the person, and which is the machine.

Turing (1950): „I believe that in about fifty years' time it will be possible to programme computers, with a storage capacity of about 109, to make them play the imitation game so well that an average interrogator will not have more than 70 % chance of making the right identification after five minutes of questioning”

So far, no success


10

More pragmatic definition

Wiener’s (1894-1964) pragmatic definition:

Intelligence is a process of acquisition and processing of information for attaining goals

Serves our purpose!

A point of departure for „constructive”, implementable intelligent systems!

Basically: the perspective adopted here


11

Towards intelligent machines

So, let us build a „machine” (hardware, software,...) that will exhibit such intelligence!

→ Artificial intelligence (term coined by John McCarthy in 1956)

→ Computational intelligence

→ Machine intelligence

→ Intelligent systems

→ Intelligent systems + soft computing


12

Artificial intelligence

What is artificial intelligence?

A modern book:


13

Russell and Norvik’s book on AI PART I: Artificial Intelligence ... 1

1 Introduction ... 3

2 Intelligent Agents ... 31

PART II: Problem-Solving ... 53

3 Solving Problems By Searching Problem-Solving Agents, Formulating Problems,Searching For Solutions, Search

Strategies,Constraint Satisfaction Search ...

4 Informed Search Methods ... 92 Best-First Search, Heuristic Functions, Memory Bounded Search, Iterative Improvement,...

5 Game Playing ... 122 Games As Search Problems, Perfect Decisions In Two-Person Games, Imperfect Decisions, Alpha-Beta Pruning, Games That Include An Element Of Chance, State-Of-The-Art Game Programs ...


14

Russell and Norvik’s book on AI PART III: Knowledge And Reasoning ... 149

6 Agents That Reason Logically ... 151

7 First-Order Logic ... 185 Syntax And Semantics, Extensions And Notational Variations,Using First-Order Logic, Representing Change In The World, Preferences Among Actions,

8 Building a Knowledge Base ... 217 Properties Of Good And Bad Knowledge Bases, Knowledge Engineering, General Ontology

9 Inference In First-Order Logic ... 265 Inference Rules Involving Quantifiers, Generalized Modus Ponens, Forward And Backward Chaining, Completeness, Resolution: Complete Inference Procedure,

10 Logical Reasoning Systems ... 297 Introduction,Indexing, Retrieval, Unification, Logic Programming Systems, Theorem Provers, Forward-Chaining Production Systems, Frame Systems, Semantic Networks, Description Logics,


15

Russell and Norvik’s book on AI PART IV: Acting Logically ... 335

11 Planning ... 337 A Simple Planning Agent,From Problem Solving To Planning, Planning In Situation Calculus, Basic Representations For Planning,Planning With Partially Instantiated Operators, Knowledge Engineering For Planning

12 Practical Planning ... 367 Practical Planners, Hierarchical Decomposition, Resource Constraints

13 Planning And Acting ... 392 Conditional Planning, A Simple Replanning Agent, Fully Integrated Planning And Execution

PART V: Uncertain Knowledge And Reasoning ... 413

14 Uncertainty ... 415 Acting Under Uncertainty,Basic Probability Notation, The Axioms Of Probability, Bayes' Rule And Its Use, Where Do Probabilities Come From?


16

Russell and Norvik’s book on AI 15 Probabilistic Reasoning Systems ... 436

Representing Knowledge In An Uncertain Domain, The Semantics Of Belief Networks, Inference In Belief Networks,inference In Multiply Connected Belief Networks, Knowledge Engineering For Uncertain Reasoning, Other Approaches To Uncertain Reasoning

16 Making Simple Decisions ... 471 Combining Beliefs And Desires Under Uncertainty, The Basis Of Utility Theory, Utility Functions, Multiattribute Utility Functions, decision Networks, The Value Of Information, Decision-Theoretic Expert Systems

17 Making Complex Decisions ... 498 Sequential Decision Problems, Value Iteration, Policy Iteration, Decision-Theoretic Agent Design, Dynamic Belief Networks, Dynamic Decision Networks...


17

Russell and Norvik’s book on AI PART VI: Learning ... 523

18 Learning From Observations ... 525 A General Model Of Learning Agents, Inductive Learning, Learning Decision Trees, Using Information Theory, Learning General Logical Descriptions ...

19 Learning In Neural And Belief Networks ... 563 How The Brain Works, Neural Networks, Perceptrons, Multilayer Feed-Forward Networks,

20 Reinforcement Learning ... 598 Passive Learning In A Known Environment, Passive Learning In An Unknown Environment, Active Learning In An Unknown Environment, Exploration, Learning An Action-Value Function, Generalization In Reinforcement Learning, Genetic and Evolutionary Programming


18

Russell and Norvik’s book on AI 21 Knowledge In Learning ... 625

Knowledge In Learning, Explanation-Based Learning, Learning Using Relevance Information, Inductive Logic Programming,

PART VII: Communicating, Perceiving, And Acting ... 649

22 Agents That Communicate ... 651 Communication As Action, A Formal Grammar For A Subset Of English,

Syntactic Analysis (Parsing), Semantic Interpretation, Ambiguity And Disambiguation ...

23 Practical Natural Language Processing ... 691 Efficient Parsing, Scaling Up The Lexicon, Scaling Up The Grammar, Ambiguity, Discourse Understanding ...

24 Perception ... 724 Image Formation, Image-Processing Operations For Early Vision, Extracting 3-D Information Using Vision, Using Vision For Manipulation And Navigation, Object Representation And Recognition, Speech Recognition ...


19

Russell and Norvik’s book on AI 25 Robotics ... 773

Tasks: What Are Robots Good For?, What Are Robots Made Of? Navigation And Motion Planning ...

PART VIII: Conclusions ... 815

26 Philosophical Foundations ... 817 The Big Questions, Foundations Of Reasoning And Perception, On The Possibility Of Achieving Intelligent Behavior, Intentionality And Consciousness,

27 AI: Present And Future ... 842 Have We Succeeded Yet?, What Exactly Are We Trying To Do?, What If We Do Succeed?


20

Russell and Norvik’s book on AI

Still in the spirit of traditional AI:

Emphasis on symbolic computation but (finally!) some numerical computations too,

Based on strict logical calculi, Based on an idealized approach to natural language, Limited use of uncertainty/imprecision calculi, Little relation to foundational works on intelligence, Little relation to real needs for useful „intelligent

systems”, in particular for decision support, Is spite of claims by traditionalists, questionable

„great successes” in terms of implementable systems.


21

What can we („soft people”) do?

So, let us try to:

Support decision making – crucial, is still a „bottleneck” in virtually all real life situations.

Take into account specifics of human beings (notably natural language!),

Use most adequate and best tools to solve the problem.

Build an implementable, intelligent decision support system!

Note: Zadeh’s recent papers on a need to a new approach to decison analysis/support using computing with words/perceptions


22

Zadeh (ca. 1995): a paradigm of computing with words (CWW), and perceptions (CWP)

Books by Zadeha and Kacprzyk (1999a, b)


23

Computing with words and percepttions

Essence:

For a human being, the only fully natural means of articulation and communication is natural language

Maybe, in many situations:

instead of traditional computing with numbers (from measurements) it would be better to compute with words (from perceptions)?

a paradigm of computing with words and perceptions (CWP)


24

Computing with words and precisiated natural languageWhat is needed for computing with words and perceptions?

A formal representation of linguistic descriptions, relations, etc.

Zadeh (early 1990s?): PNL (precisiated natural language)

PNL: a subset of natural language that is equipped with a constraint-centered semantics, and associated with a generalized constraint language

A set of so-called generalized constraints corresponding to linguistic statementsX isr R - assigns a value


25

Various forms of generalized constraints

Examples of constraints:

Equality: X is= R (X=R) Possibilistic constraint: X is R (R is a possibilistic distribution) Probabilistic constraint: X isp R (R is a probabilistic distribution) Usuality constraints: X isu R [usually(X is R)] Veristic, rough set, etc.

Here: mainly the usuality constraint in most, almost all, much more than 50%, ... cases

because we seek some „regularities”, „normal/typical” relations i.e. those which „usually happen”For instance, commonsense (world) knowldege


26

Decision making and decision support

Point of departure: decision makingOmnipresent!

First formal attempts: a structured problem:

Set of options, A preference structure (utility function), A best decision is chosen (optimization)

All this well defined!


27

Recent trends:

Modern, good, ... decision making (good decisions!)

Decision making process (DMP):

Use of own and external knowledge, Involvement of various „actors”, aspects, etc. Individual habitual domains (P.L. Yu), Non-trivial rationality, Different paradigms when appropriate.


28

Soft approach to systems analysis

For instance:

Peter Checkland’s (1975-99) soft approach to systems analysis

Deliberative (soft) decision making:

To perceive the whole picture, To observe it from all angles (actors, criteria,...) To find a good decision using knowledge and intuition.

Intelligent systems + soft computing!Most elements may only be expressed in natural language!


29

Decision making process:modern approachesModern decision making process (involves creative,

strategic, deliberative, etc. decision making):

Recognition, Deliberation and analysis, Gestation and enligtenment („eureka!”, „aha”), Rationalization, Implementation.

Involves much intelligence!Implementable only through intelligent systems!


30

Modern decision making paradigm (continued): Heavily based on data, information and knowledge, and

human specific characteristics (intuition, attitude, natural language for communication and articulation,...)

Need number crunching, but also more „delicate” and sophisticated „intelligent” analyses,

Heavily relying on computer systems, and capable of a synergistic human-computer interaction, notably using (quasi)natural language.

So:

Intelligent decision support systems!+ soft computing+ (quasi)natural language


31

DSSs – characteristic features:

Emphasis on:

Ill/semi/un-structured questions and problems, Non-routine, one of a kind answers, A flexible combination of analytical models and data, Various kinds of data, e.g. numeric, textual, verbal,... Interactive interface (e.g. GUI, LUI), Iterative operation („what if”), Supporting various decision making styles, Supporting alternate decision making passes, ...

Intelligent systems!!!Knowledge based!


32

Data, information and knowledge

Various definitions:

data - raw facts; information - data in a context relevant to an individual,

team or organization, knowledge - an individual’s utilization of information and

data complemented by an unarticulated expertise, skills, competencies, intuitions, experience and motivations.

So:

knowledge resides in an individual person and not in a collection of information (Churchman, 1970s)

We gain knowledge through communication, personal interactions and “bouncing ideas” off other people.


33

Knowledge:

Explicit, expressed in words or numbers, and shared as data, equations, specifications, documents, and reports; can be transmitted individuals and formally recorded,

Tacit, highly personal, hard to formalize, and difficult to communicate or share with others; technical (skills or crafts), and cognitive (perceptions, values, beliefs, and mental models).

Both extremely relevant for intelligent systems (e.g. decision support)!

Notice: Zadeh’s computing with words and perceptions!


34

Brief history of DSSs:

Starting point:

Mid-1960s: development of IBM 360 and a wider use of distributed, time-sharing computing

Mid-1960s: MISs (management information systems) first to provide managers with structured, periodic reports,

Late 1960s-early 1970s: attempts to use analytical models, first attempts at interactive systems


35

Brief history of DSSs:

Early 1980s:

EISs (executive information systems) that use relational database, and use predefined screans, and are made by analysts for executives,

knowledge-oriented DSSs (use of AI tools), group DSSs,


36

A technology shift:

Early 1990s:

Use of relational DBMS techniques, Shift from mainframe based to client-server based

solutions, Object oriented technology for builing „reusable” systems.

Mid-1990s: Data warehouses and on line analytical processing

(OLAP) tools, Web based and Web enabled systems


37

Basic types of DSSs:

Data driven, Communication driven and group DSSs, Document driven, Model driven, Knowledge driven, Web based and interorganizational.


38

Specifics of DSSs:

Data Driven DSSs - emphasize access to and manipulation of internal company data and sometimes external data. Low level: simple file systems with query and retrieval tools, then data warehouses, finally with On-line Analytical Processing (OLAP) or data mining tools.

Communications Driven DSSs - use network and communications technologies to facilitate collaboration and communication,

Group GDSSs - interactive, computer-based systems that facilitate solution of unstructured problems by a set of decision-makers working together as a group.

Document Driven DSSs - integrate a variety of storage and processing technologies for a complete document retrieval and analysis; documents may contain numbers, text, multimedia.

Model Driven DSSs -emphasize access to and manipulation of a model, e.g., statistical, financial, optimization and/or simulation; use data and parameters, but are not usually data intensive.

Knowledge Driven DSSs – interactive systems with specialized problem-solving expertise consisting of knowledge about a particular domain, understanding of problems within that domain, and "skill" at solving some of these problems.

Web based DSSs – computerized system that deliver decision support related information and/or tools to a manager/analyst using a "thin-client" Web browser (Explorer); TCP/IP protocol!


39

Types of DSSs and the role of Web mining (3): Data Driven DSSs - emphasize access to and manipulation of

internal company data and sometimes external data. Low level: simple file systems with query and retrieval tools, then data warehouses, finally with On-line Analytical Processing (OLAP) or data mining tools.

Tendency: „intelligent”, soft computing and natural language

Examples:

Fuzzy querying, also over the Internet: FQUERY for Access (Kacprzyk and Zadrożny, 1996 - ...),

Linguistic data summaries (Kacprzyk and Zadrożny, 1998 -...) Fuzzy logic + OLAP (Laurent, 2001)


40

Concept of a linguistic data(base) summary

Yager’s approach [Yager(1982, ...), ...., Kacprzyk and Yager(2001)]:

- V - a quality of interest, e.g. salary in a database of workers,

- Y={y1,...,yn} – a set of objects (records) that manifest V, e.g. the set of workers; V(yi) are values of quality V for object yi.

A linguistic summary of a data set consists of:

a summarizer S (e.g. young), a quantity in agreement Q (e.g. most), truth (validity) T - e.g. 0.7,

e.g., T(most of employees are young)=0.7


41

Yager’s linguistic data(base) summaries:

Yager’s concept of a linguistic summary: Intuitive, Simple, Extendable (Kacprzyk and Yager, 2001), Implementable, ...

But: Concerns relational (numerical) databases! structured data!


42

Example of implementation

A small computer retailer in South Poland: Owner:

must make sophisticated decisions concerning:

number of employees on Saturday, type of advaertisement, Commisions

But: very busy

Simple summaries, in natural language! Inexpensive technology, add-in without any „touching” his database!


43

Example:...

Relations between commision and type of product:

So: No problem with accessories and network elements, Critical are: elements, software and computers!

About 1/2 of sales of network elements is with a high ommission

Much sales of accessories is with a high commission

Much sales of elements is with a low commission

About 1/2 sales of software is with a low commission

About 1/2 sales of computers is with a low commision


44

Extensions (external data from WWW)

Own database only!

But: a company operates in an environment (e.g. weather) So, e.g., lrelations between group of products, time of sale,

temperature, precipitacion, and type of customers:

Next step: semi-structured weather info (text forecasts from a local newspaper, SMS messages from a local provider) – local info!

Very few sales of software in hot days to individual customers

About 1/2 of sales of accessories in rainy days on weekends by the end of the year

About 1/3 of sales of computers in rainy days to individual customers


45

Conclusions

My purpose:

To present and advocate an urgent need for intelligent systems,

To advocate the need to employ a broader perspective on intelligence and intelligent systems than it is customary in traditional artificial intelligence,

To advocate the need to more adequately, effectively and efficiently deal with natural language,

To sketch Zadeh’s computing with words and perceptions as a viable, simple and intuitive alternative for the above,

To show an ideaq of an implemented „intelligent” system for supporting decisions in a company.

seminar on new trends in intelligent systems and soft computing october 2-3.2003, granada, spain 1...

Documents