8/5/2014

1

INTRODUCTION TO SOFT COMPUTING

Harshali Patil

Introduction

� Evolution of Computing

� Soft computing constituents

� From conventional AI to Computational Intelligence

� Machine Learning basics

8/5/2014

2

Problem Solving

2) How to reuse the previous knowledge?

3) Proposed solution is

valid or not?

1) What kind of similar

problems you have solved?

4) Is it necessary to

retain the knowledge?

New

Problem

Reasoning Solution

Case Based Reasoning

8/5/2014

3

Artificial Intelligence

� If intelligence can be induced in machines it is called as artificial intelligence.

� Soft computing is a part of artificial intelligent techniques

� Closed related to machine intelligence/computational intelligence

Soft Computing

Neural Networks

Fuzzy Inferencesystems

Neuro-Fuzzy

Computing

Derivative-Free

OptimizationSoft ComputingSoft ComputingSoft ComputingSoft Computing

+ =

8/5/2014

4

What is Soft Computing?

� The idea behind soft computing is to model cognitive behaviour of human mind.

� Soft computing is foundation of conceptual intelligence in machines.

� Unlike hard computing , Soft computing is tolerant of imprecision, uncertainty, partial truth, and approximation.

Soft Computing

AccordingAccordingAccordingAccording totototo ProfProfProfProf.... ZadehZadehZadehZadeh::::

"...in contrast to traditional hardcomputing, soft computing exploits thetolerance for imprecision, uncertainty, andpartial truth to achieve tractability,robustness, low solution-cost, and betterrapport with reality”

8/5/2014

5

Soft Vs Hard Computing

Symbolic Logic

Reasoning

Traditional NumericalModeling and Search

Approximate Reasoning

Functional Approximationand Randomized

Search

HARD COMPUTING SOFT COMPUTING

Precise Models Approximate Models

Hard ComputingHard ComputingHard ComputingHard Computing Soft ComputingSoft ComputingSoft ComputingSoft Computing

Conventional computing requires a

precisely stated analytical model.

Soft computing is tolerant of

imprecision.

Often requires a lot of computation

time.

Can solve some real world problems in

reasonably less time.

Not suited for real world problems for

which ideal model is not present.

Suitable for real world problems.

It requires full truth Can work with partial truth

It is precise and accurate Imprecise.

High cost for solution Low cost for solution

8/5/2014

6

Overview techniques of SC

Neural Networks

Fuzzy Logic

Genetic Algorithm

Hybrid Systems

Evolution of Computing

1940s 1947

Cybemetics

1943 McCulloach

Pitts neuron model

1950s 1956 AI 1957 Perceptron

1960s 1960 LISP

language

1960 Adaline

Masaline

1965 Fuzzy

sets

1970s Mid 1970s

knowledge

engineering

(expert system)

1974 Birth of back

propogation

algorithm

1975 Cognitron

Neocognitron

1974 Fuzzy

controller

1970s

Genetic

algorithm

1980s 1980 self organizing

map

1982 hopfield net

1983 Boltzmann

machine

1986

Backpropogation

algorithm boom

1985 Fuzzy

modelling

Mid 1980s

Artificial life

Immune

modelling

1990s 1990s Neuro

fuzzy modelling

1991 ANFIS

1990 Genetic

programming

8/5/2014

7

SC constituents (the first three items) and conventional AI

MethodologyMethodologyMethodologyMethodology StrengthStrengthStrengthStrength

Neural network Learning and adaptation

Fuzzy set theory Knowledge representation via fuzzy if-then rules

Genetic algorithm and simulated annealing

Systematic random search

Conventional AI Symbolic manipulation

Character recognizer

8/5/2014

8

Features of Conventional AI

�Conventional AI manipulates symbols on the assumption that human intelligence behavior can be stored in symbolically structured knowledge bases: this is known as: “ The physical symbol system hypothesis”

�The knowledge-based system (or expert system) is an example of the most successful conventional AI product

What is expert system?

� An expert systemexpert systemexpert systemexpert system is software that uses a knowledge base of human expertise for problem solving, or to clarify uncertainties where normally one or more human experts would need to be consulted

8/5/2014

9

Expert system

Building blocks of expert system

� Knowledge base: Knowledge base: Knowledge base: Knowledge base: factual knowledge and heuristic knowledge

� Knowledge representation: Knowledge representation: Knowledge representation: Knowledge representation: in the form of rules

� Problem solving model: Problem solving model: Problem solving model: Problem solving model: forward chaining or backward chaining

� Note:-

� Knowledge engineering:- building an expert system

� Knowledge engineers:- practitioners.

8/5/2014

10

Definitions of AI

� “AI is the study of agents that exists in an environment and perceive and act” [S. Russel & P. Norvig]

� “AI is the act of making computers do smart things” [Waldrop]

� “AI is a programming style, where programs operate on data according to rules in order to accomplish goals” [W.A. Taylor]

Definitions of AI

� “AI is the activity of providing such machines as computers with the ability to display behavior that would be regarded as intelligent if it were observed in humans” [R. Mc Leod]

� “Expert system is a computer program using expert knowledge to attain high levels of performance in a narrow problem area” [D.A. Waterman]

8/5/2014

11

Definitions of AI

� “Expert system is a caricature of the human expert, in the sense that it knows almost everything about almost nothing” [A.R. Mirzai]

� AI is changing rapidly, these definitions are already obsolete!

Applications of expert system

� Diagnosis and Troubleshooting of Devices and Systems of All Kinds

� Planning and Scheduling

� Configuration of Manufactured Objects from Subassemblies

� Financial Decision Making

� Knowledge Publishing

� Design and Manufacturing

8/5/2014

12

If the facts don't fit the If the facts don't fit the If the facts don't fit the If the facts don't fit the theory, change the facts. theory, change the facts. theory, change the facts. theory, change the facts.

- Albert Einstein

GEMS CBR for Remote Diagnostics

Information

System

Servers

GE RegionalService Team

DiagnosticsSpecialist

ServiceEngineer

Phone, E-mail

FAX, Web

On-site Monitoring

Remote Data

Access

PartsSpecialist

TechnicalAnswer Center

• Call Management/ Commitment Tracking System

• Monitoring & Diagnostics

• Problem/Solution DB• CBR using DB

8/5/2014

13

Machine Learning basics

� Machine learning, a branch of artificial intelligence, concerns the construction and study of systems that can learn from data.

� E.g. a machine learning system could be trained on email messages to learn to distinguish between spam and non-spam messages. After learning, it can then be used to classify new email messages into spam and non-spam folders.

Research Areas

Intelligent RobotsIntelligent RobotsIntelligent RobotsIntelligent Robots� Bayesian inference and design technique for uncertain scene recognition�Combination Image filtering and Bayesian inference�Navigation technique research for autonomous mobile robot�Evolving a mobile robot controller

8/5/2014

14

Research Areas

Intelligent AgentIntelligent AgentIntelligent AgentIntelligent Agent� Intelligent virtual secretary agent�Conversational agent�Intelligent assistants for smart phone service

Research Areas

BioinformaticsBioinformaticsBioinformaticsBioinformatics�Bioinformatics: the collection, classification, storage, and analysis of biochemical and biological information using computers especially as applied in molecular genetics and genomics�Classification techniques in Bioinformatics�Agent driven virtual cell modelling

8/5/2014

15

Research Areas

UbiquitousUbiquitousUbiquitousUbiquitous�Developing an adaptation scheme in the context of middleware and applications�Developing context-aware system for ubiquitous systems�Developing basic theories and algorithms of the advanced intelligent models for ubiquitous environment

Research Areas

Intrusion Detection System (IDS)Intrusion Detection System (IDS)Intrusion Detection System (IDS)Intrusion Detection System (IDS)�HMM-based intrusion detection system�Generation of various intrusion patterns using interactive genetic algorithm�Viterbi algorithm for intrusion type identification�Rule-based integration of multiple measure-models

8/5/2014

16

Research Areas

BiometricsBiometricsBiometricsBiometrics�Analysis and evaluation techniques of fingerprint recognition system�Development of classification and matching algorithm for fingerprint recognition

Neuro Fuzzy and Soft Computing Characteristics

� With NF modeling as a backbone, SC can be characterized as:

� Human expertise (fuzzy if-then rules)

� Biologically inspired computing models (NN)

� New optimization techniques (GA, SA, RA)

� Numerical computation (no symbolic AI so far, only numerical)

� New application domains: mostly computation intensive like adaptive signal processing, adaptive control, nonlinear system identification etc

8/5/2014

17

Neuro Fuzzy and Soft Computing Characteristics

� Model free learning:-models are constructed based on the target system only

� Intensive computation: based more on computation

� Fault tolerance: deletion of a neuron or a rule does not destroy the system. The system performs with lesser quality

� Goal driven characteristics:- only the goal is important and not the path.

� Real world application:- large scale, uncertainties

Neural Network

� DARPA Neural Network Study (1988, AFCEA DARPA Neural Network Study (1988, AFCEA DARPA Neural Network Study (1988, AFCEA DARPA Neural Network Study (1988, AFCEA International Press, p. 60): International Press, p. 60): International Press, p. 60): International Press, p. 60):

... a neural network is a system composed of many simple processing elements operating in parallel whose function is determined by network structure, connection strengths, and the processing performed at computing elements or nodes.

8/5/2014

18

Definition of Neural Network

� According to According to According to According to HaykinHaykinHaykinHaykin (1994), p. 2: (1994), p. 2: (1994), p. 2: (1994), p. 2:

A neural network is a massively parallel distributed processor that has a natural propensity for storing experiential knowledge and making it available for use. It resembles the brain in two respects:

� Knowledge is acquired by the network through a learning process.

� Interneuron connection strengths known as synaptic weights are used to store the knowledge

� According to According to According to According to NigrinNigrinNigrinNigrin (1993), p. 11: (1993), p. 11: (1993), p. 11: (1993), p. 11:

A neural network is a circuit composed of a very large number of simple processing elements that are neurally based. Each element operates only on local information.

Furthermore each element operates asynchronously; thus there is no overall system clock.

8/5/2014

19

� According to According to According to According to ZuradaZuradaZuradaZurada (1992): (1992): (1992): (1992):

Artificial neural systems, or neural networks, are physical cellular systems which can acquire, store and utilize experiential knowledge.

Multi disciplinary view of Neural Networks

8/5/2014

20

Fuzzy Logic

� Origins: Multivalued Logic for treatment of imprecision Origins: Multivalued Logic for treatment of imprecision Origins: Multivalued Logic for treatment of imprecision Origins: Multivalued Logic for treatment of imprecision and vagueness and vagueness and vagueness and vagueness

� 1930s: Post, Kleene, and Lukasiewicz attempted to represent undetermined, unknown, and other possible intermediate truth-values.

� 1937: Max Black suggested the use of a consistency profile to represent vague (ambiguous) concepts.

� 1965: Zadeh proposed a complete theory of fuzzy sets (and its isomorphic fuzzy logic), to represent and manipulate ill-defined concepts.

FUZZY LOGIC – LINGUISTIC VARIABLES

� Fuzzy logic gives us a language (with syntax and local semantics) in which we can translate our qualitative domain knowledge.

� Linguistic variables to model dynamic systems

� These variables take linguistic values that are characterized by:

� a label - a sentence generated from the syntax � a meaning - a membership function determined by a local

semantic procedure

8/5/2014

21

Linguistic variables

� Linguistic variables associate a linguistic condition with a crisp variable.

� A crispcrispcrispcrisp variable is the kind of variable that is used in most computer programs: an absolute value.

� A linguistic variablelinguistic variablelinguistic variablelinguistic variable, on the other hand, has a proportional nature: in all of the software implementations of linguistic variables, they are represented by fractional values in the range of 0 to 1.

Linguistic variables

Linguistic variables in soup instructions

8/5/2014

22

An old friend comes into your shop asking to buy a few widgets, and wants your best price. The onus is on you to come up with a price given many parameters. Taking this hypothetical case we need to account for:

• Cost of the widgets • Normal markup • Shelf time of the product • Shelf life of the product • Length of the relationship • Customer payment history • Quantity of the sale • Repeat business potential

Parameters to consider pricing a widgetParameters to consider pricing a widgetParameters to consider pricing a widgetParameters to consider pricing a widget

8/5/2014

23

Figure: Linguistic variable HOTFigure: Linguistic variable HOTFigure: Linguistic variable HOTFigure: Linguistic variable HOT

8/5/2014

24

Figure: F_OR operator (Fuzzy OR)

Figure: F_EQ (Fuzzy equal)

8/5/2014

25

FUZZY LOGIC – REASONING METHODS

� The meaning of a linguistic variable may be interpreted as an elastic constraint on its value.

� These constraints are propagated by fuzzy inference operations, based on the generalized modus-ponens.

� An FL Controller (FLC) applies this reasoning system to a Knowledge Base (KB) containing the problem domain heuristics.

� The inference is the result of interpolating among the outputs of all relevant rules.

� The outcome is a membership distribution on the output space, which is defuzzified to produce a crisp output.

� There are two consistent logical argument constructions: modus ponens ("the way that affirms by affirming") and modus tollens ("the way that denies by denying"). Here are how they are constructed:� Modus Ponens: "If A is true, then B is true. A is true. Therefore, B is true."� Modus Tollens: "If A is true, then B is true. B is not true. Therefore, A is

not true.“

� There are two related incorrect and inconsist constructions: affirming the consequent and denying the antecedent.� Affirming the Consequent: "If A is true, then B is true. B is true.

Therefore, A is true."� Denying the Antecedent: "If A is true, then B is true. A is not true.

Therefore, B is not true."

8/5/2014

26

Example

Here is a sensible example, illustrating each of the above:

� "If it is a car, then it has wheels. It is a car. Therefore, it has wheels." (Modus Ponens -CORRECT)

� "If it is a car, then it has wheels. It does not have wheels. Therefore, it is not a car." (Modus Tollens -CORRECT)

� "If it is a car, then it has wheels. It has wheels. Therefore, it is a car." (Affirming the Consequent -INCORRECT.)� Comment: why is this incorrect? Well, the thing might

have wheels but that doesn't mean it has to be a car. It might be a cart, or rollerblades, or a moped. It doesn't have to be a car.

� "If it is a car, then it has wheels. It is not a car. Therefore, it does not have wheels." (Denying the Antecedent - INCORRECT)� Comment: why is this incorrect? Consider

the argument for the "affirming the consequent" example. Rollerblades are not cars, but they DO have wheels.

8/5/2014

27

Genetic algorithm

EVOLUTIONARY PROCESS

Steps involved in Genetic Algorithm

� The genetic algorithms follow the evolution process in the nature to find the better solutions of some complicated problems. Foundations of genetic algorithms are given in Holland (1975) and Goldberg (1989) books.

� Genetic algorithms consist the following steps:

� Initialization� Selection� Reproduction with crossover and mutation

� Selection and reproduction are repeated for each generation until a solution is reached.

� During this procedure a certain strings of symbols, known as chromosomes, evaluate toward better solution.

8/5/2014

28

Hybrid Systems

� Hybrid systems enables one to combine various soft computing paradigms and result in a best solution. The major three hybrid systems are as follows:

� Hybrid Fuzzy Logic (FL) Systems

� Hybrid Neural Network (NN) Systems

� Hybrid Evolutionary Algorithm (EA) Systems

Applications of Soft Computing

� Handwriting Recognition� Image Processing and Data Compression� Automotive Systems and Manufacturing� Soft Computing to Architecture� Decision-support Systems� Soft Computing to Power Systems� Neuro Fuzzy systems� Fuzzy Logic Control� Machine Learning Applications� Speech and Vision Recognition Systems� Process Control and So on

8/5/2014

29