A

Seminar Presentation Report

On

“ARTIFICIAL NEURAL NETWORK”

Submitted to: Submitted by:

Er. Rashi Gupta Balveer Singh

(CSE ) Final Year (CSE)

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

MARWAR ENGINEERING COLLEGE AND RESEARCH CENTER JODHPUR (RAJ.)

MARWAR ENGINEERING COLLEGE & RESEARCH CENTRE JODHPUR

CERTIFICATE

This is to certify that the project entitled “ARTIFICIAL NEURAL NETWORK “has been

carried out by BALVEER SINGH under my guidance in partial fulfillment of the degree of

Bachelor of Engineering in Computer Engineering Rajasthan Technical University, Kota during

the academic year 2009-2010. To the best of my knowledge and belief this work has not been

submitted elsewhere for the award of any other degree.

Head of the Department

AFTIFICIAL NEURAL NETWORK

Definition:

An Artificial Neural Network (ANN) is an information-processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurons.

Introduction:

ANNS are used as a solution to various problems; however, their success as an intelligent pattern recognition methodology has been most prominently advertised. The most important, and attractive, feature of ANNs is their capability of learning (generalizing) from example (extracting knowledge from data). ANNs can do this without any pre-specified rules that define intelligence or represent an expert’s knowledge. This feature makes ANNs a very popular choice for gene expression analysis and sequencing. Due to their power and flexibility, ANNS have even been used as tools for relevant variable selection, which can in turn greatly increase the expert’s knowledge and understanding of the problem.

A neural network is characterized by its pattern of connections between the neurons referred to as network architecture and its method of determining the weights on the connections called training or learning algorithm. The weights are adjusted on the basis of data. In other words, neural networks learn from examples and exhibit some capability for generalization beyond the training data. This feature makes such computational models very appealing in application domains where one has little or incomplete understanding of the problem to be solved, but where training data is readily available. Neural networks normally have great potential for parallelism, since the computations of the components are largely interdependent of each other.

Artificial neural networks are viable computational models for a wide variety of problems. Already, useful applications have been designed, built, and commercialized for various areas in engineering, business and biology. These include pattern classification, speech synthesis and recognition, adaptive interfaces between human and complex physical systems, function approximation, image compression, associative memory, clustering, forecasting and prediction, combinatorial optimization, nonlinear system modeling, and control. Although they may have been inspired by neuroscience, the majority of the networks have close relevance or counterparts to traditional statistical methods such as non-parametric pattern classifiers, clustering algorithm, nonlinear filters, and statistical regression models

NEURAL NETWORKS

Artificial neural networks have emerged from the studies of how brain performs. The human brain consists of many millions of individual processing elements, called neurons that are highly interconnected.

Information from the outputs of the neurons, in the form of electric pulses is received by the cells at connections called synapses. The synapses connect to the cell inputs, or dendrites and the single output pf the neuron appears at the axon. An electric pulse is sent down the axon when the total input stimuli for all of the dendrites exceed a certain threshold.

Artificial neural networks are made up of simplified individual models of the biological neuron that are connected together to form a network. Information is stored in the network in the form of weights or different connections strengths associated with synapses in the artificial neuron models.

Many different types of neural networks are available and multi layer neural networks are the most popular which are extremely successful in pattern reorganization problems. An artificial neuron model is shown below. Each neuron input is weighted by W. changing the weights of an element will alter the behavior of the whole network. The output y is obtained by summing the weighted inputs to the neuron and passing the result through a non-linear activation function, f ().

Multi layer networks consists of an input layer, a hidden layer are made up of no. of nodes. Data flows through the network in one direction only, from input to output; hence this type of network is called a feed-forwarded network. A two-layered network is shown below.

NEURAL NETWORKS IN PROCESS CONTROL

Artificial neural networks are implemented as software packages in computers and being used to incorporate of artificial intelligence in control system. ANN is basically mathematical tools which are being designed to employ principles similar to neurons networks of biological system. ANN is able to emulate the information processing capabilities of biological neural system. ANN has overcome many of the difficulties that t conventional adaptive control systems suffer while dealing with non linear behavior of process.

PROCEDURES FOR ANN SYSTEM ENGINEERING

In realistic application the design of ANN system is complex, usually iterative and interactive task. Although it is impossible to provide an all inclusive algorithmic procedure, the following highly interrelated, skeletal steps reflect typical efforts and concerns. The plethora of possible ANN design parameters include:

• The interconnection strategy/network topology/network structure.

• Unit characteristics (may vary within the network and within subdivisions within the network such as layers).

• Training procedures.

• Training and test sets.

• Input/output representation and pre- and post-processing.

FEATURES OF ANN

• Their ability to represent nonlinear relations makes them well suited for non linear modeling in control systems.

• Adaptation and learning in uncertain system through off line and on line weight adaptation

• Parallel processing architecture allows fast processing for large-scale dynamic system.

• Neural network can handle large number of inputs and can have many outputs.

Neural network architecture have learning algorithm associated with them. The most popular network architecture used for control purpose is multi layered neural network [MLNN] with error propagation [EBP] algorithm.

BENEFITS OF NEURAL NETWORKS

It is apparent that a neural network derives its computational power through, first, its massively parallel distributed structure and, second its ability to learn and therefore generalize. Generalization refers to the neural network producing reasonable outputs for inputs not encountered during training. These two information-processing capabilities make it possible for neural networks to solve complex problems that are currently intractable. A complex problem of interest is decomposed into a number of relatively simple tasks, and neural networks are assigned a subset of tasks that match their inherent capabilities. The use of neural networks offers the following useful properties and capabilities.

• Nonlinearity, An artificial neuron can be linear or nonlinear. A neural network made up of an interconnection of nonlinear neurons, is itself nonlinear. Nonlinearity is a highly important property, particularly if the underlying physical mechanism responsible for generation of the input signal (e.g., speech signal) is inherently nonlinear.

• Input-Output Mapping, A popular paradigm of learning called learning with a teacher or supervised learning involves modification of the synaptic weights of a neural network by applying a set of labeled training samples or task examples. Each example consists of a unique input signal and a corresponding desired response. The network is presented with an example picked at random from the set, and the synaptic weights of the network are modified to minimize the difference between the desired response and the actual response of the network produced by the input signal in accordance with an appropriate statistical criterion. Thus the network learns from the examples by constructing an input-output mapping for the problem at hand.

• Adaptivity, Neural networks have a built-in capability to adapt their synaptic weights to changes in the surrounding environment. In particular, a neural network trained to operate in a specific environment can be easily restrained to deal with minor changes in the operating environmental conditions. Moreover, when it is operating in a non-stationary environment, a neural network can be designed to change its synaptic weights in real time.

• Evidential Response, In the context of pattern classification, a neural network can be designed to provide information not only about which pattern to select, but also about the confidence in the decision made. This latter information may be used to reject ambiguous patterns, should they rise, and thereby improving the classification performance of the network.

• Contextual Information, Knowledge is represented by the very structure and activation state of a neural network. Every neuron in the network is potentially affected by the global activity of all other neurons in the network. Consequently, contextual information is dealt with naturally by a neural network.

• Fault Tolerance, A neural network implemented in hardware form, has the potential to be inherently fault tolerant, or capable of robust computation, in the sense that its performance degrades gracefully under adverse operating conditions. This can be attributed to its massively distributed nature of information stored in the network. Thus, in principle, a neural network exhibits a graceful degradation in performance rather than catastrophic failure.

• VLSI Implement ability, The massively parallel nature of a neural network makes it potentially fast for the computation of certain tasks. This same feature makes a neural network well suited for implementation using very-large-scale-integrated technology.

One particular beneficial virtue of VLSI is that it provides a means of capturing truly complex behavior in a highly hierarchical fashion.

• Neurobiological Analogy, The design of a neural network is motivated by analogy with the brain, which is living proof that fault tolerant parallel processing is not only physically possible but also fast and powerful.

NEURAL NETWORK FOUNDATIONS

Artificial Neural networks (ANNs) belong to the adaptive class of techniques in the

machine learning arena. ANNS are used as a solution to various problems; however, their

success as an intelligent pattern recognition methodology has been most prominently

advertised. Most models of ANNs are organized in the form of a number of processing units

called artificial neurons, or simply neurons, and a number of weighted connections referred to

as artificial synapses between the neurons. The process of building an ANN, similar to its

biological inspiration, involves a learning episode. During learning episode, the network

observes a sequence of recorded data, and adjusts the strength of its synapses according to a

learning algorithm and based on the observed data. The process of adjusting the synaptic

strengths in order to be able to accomplish a certain task, much like the brain, is called

“learning”. Learning algorithms are generally divided into two types, supervised and

unsupervised. The supervised algorithms require labeled training data. In other words, they

require more a priori knowledge about the training set.

MODELS OF A NEURON

A neuron is an information-processing unit that is fundamental to the operation of a

neural network. Fig.2.1a shows the model of a neuron, which forms the basis for designing

artificial neural networks. The three basic elements of the neuronal model are

1. A set of synapses or connecting links, each of which is characterized by a weight or

strength of its own. Specifically, a signal xj at which the input of synapse j connected

to neuron k is multiplied by the synaptic weight wjk.

2. An adder for summing the input signals, weighted by the respective synapses of the

neuron; the operations described here constitutes a linear combiner.

3. An activation function for limiting the amplitude of the output of a neuron. The

activation is also referred to as a squashing function as it squashes the permissible

amplitude range of the output signal to some finite value.

NETWORK ARCHITECTURES

The manner in which the neurons of a neuron network are structured is intimately

linked with the learning algorithm used to train the network. Network structure can be broadly

divided into three classes of network architectures.

Single-Layer Feed forward Networks

In a layered neural network the neurons are organized in the form of layers. The

simplest form of a layered network consists of an input layer of source nodes those projects

onto an output layer of neurons, but not vice-versa. In other words, this network is strictly a

feed forward or acyclic type.

Such a network is called a single-layer network, with the designation “single layer”

referring to the output layer of computational nodes. The input layer of source nodes are not

counted as no computation is performed here.

Multilayer Feed forward Networks

The second class of a feed forward neural network distinguishes itself by the

presence of one or more hidden layers, whose computation nodes are correspondingly called

hidden neurons or hidden units. The function of hidden neurons is to intervene between the

external input and the network output in some useful manner. By adding one or more hidden

layers, the network is enabled to extract higher-order statistics.

In a rather loose sense the network acquires a global perspective despite its local

connections due to the extra set of synaptic connections and extra dimension of neural

interactions. The ability of hidden neurons to extract higher-order statistics is particularly

valuable when the size of the input is large.

Recurrent Networks

A recurrent network distinguishes itself from a forward neural network in that it has

at least one feedback loop. For example, a recurrent network may consist of a single layer of

neurons with each neuron feeding its output signal back to the inputs of all the other neurons,

as illustrated in the architectural graph. The presence of feedback loops has a profound impact

on the learning compatibility of the network and its performance. Moreover, the feedback

loops involve the use of particular branches composed of unit-delay elements (denoted by z -1),

which result in a nonlinear dynamical behavior, assuming that the neural network contains

nonlinear units.

TRAINING OF NEURAL NETWORKS

Neural networks are models that may be used to approximate summaries, classify,

generalize or otherwise represent real situations. Before models can be used they have to be

trained or made to ‘fit’ into the representative data. The model parameters, e.g., number of

layers, number of units in each layer and weights of the connections between them, must be

determined. In ordinary statistical terms this is called regression. There are two fundamental

types of training with neural networks: supervised and unsupervised learning. For supervised

training, as in regression, data used for the training consists of independent variables (also

referred to as feature variables or predictor variables) and dependent variables (target values).

The independent variables (input to the neural network) are used to predict the dependent

variables (output from the network). Unsupervised training does not have dependent (target)

values supplied: the network is supposed to cluster the data automatically into meaningful sets.

The fundamental idea behind training, for all neural networks, is in picking a set of

weights and then applying the inputs to the network and gauging the network’s performance

with this set of weights. If the network does not perform well, then the weights are modified

by an algorithm specific to each architecture and the procedure is then repeated. This iterative

process continues until some pre-specified criterion has been achieved. A training pass

through all vectors of the input data is called an epoch. Iterative changes can be made to the

weights with each input vector, or changes can be made after all input vectors have been

processed. Typically, weights are iteratively modified by epochs.

LEARNING TECHNIQUES

Learning rules are algorithm for slowly alerting the connections weighs to achieve a desirable

goal such a minimization of an error function. The generalized step for any neural network

leaning algorithm is follows are the commonly used learning algorithm for neural networks.

Multi layer neural net (MLNN)

Error back propagation (EBB)

Radial basis functions (RBF

Reinforcement learning

Temporal deference learning

Adaptive resonance theory (ART)

Genetic algorithm

Selection of a particular learning algorithm depends on the network and network

topology. As MLNN with EBP is most extensively used and widely accepted network for

process application, namely for identification and control of the process.

MLNN IN SYSTEM IDENTIFICATION:

There has been an “explosion” of neural network application in the areas

of process control engineering in the last few years. Since it become very difficult to obtain the

model of complex non-linear system due its unknown dynamics and a noise environment. it

necessitates the requirement for a non classic technique which has the ability to model the

physical process accurately. since nonlinear governing relationships can be handled very

contendly by neutral network, these networks offer a cost effective solution to modeling of time

varying chemical process.

Using ANN carries out the modeling of the process by using ANN by any one of the following

two ways:

Forward modeling

Direct inverse modeling

FORWARD MODELING

The basic configuration used for non-linear system modeling and identification using

neural network. The number of input nodes specifies the dimensions of the network input. In

system identification context, the assignment of network input and output to network input

vector.

DIRECT INVERSE MODELING:

This approach employs a generalized model suggested by Psaltis et al.to learn the inverse

dynamic model of the plant as a feed forward controller. Here, during the training stage, the

control input are chosen randomly within there working range. And the corresponding plant

output values are stored, as a training of the controller cannot guarantee the inclusion of all

possible situations that may occur in future. Thus, the developed model has take of robustness

The design of the identification experiment used to guarantee data for training the neural

network models is crucial, particularly, in-linear problem. The training data must contain process

input-output information over the entire operating range. In such experiment, the types of

manipulated variables used are very important.

The traditional pseudo binary sequence (PRBS) is inadequate because the training data

set contains most of its steady state information at only two levels, allowing only to fit linear

model in over to overcome the problem with binary signal and to provide data points throughout

the range of manipulated variables. The PRBS must be a multilevel sequence. This kind of

modeling of the process play a vital role in ANN based direct inverse control configuration.

ANN BASED CONTROL CONFIGURATION:

Direct inverse control

Direct adaptive control

Indirect adaptive control

Internal model control

Model reference adaptive control

DIRECT INVERSE CONTROL

This control configuration used the inverse planet model. Fro the direct inverse control.

The network is required to be trained offline to learn the inverse dynamics of the plant. The

networks are usually trained using the output errors of the networks and not that of the plant. The

output error of the networks is defined.

En=1/2(ud-on)2

Where En is the networks output error ud is the actual controls signal required to get desired

process output and on is the networks output. When the networks is to be trained as a controller.

The output errors of the networks are unknown. Once the network is trained using direct inverse

modeling learns the inverse system model. It is directly placed in series with the plant to be

controlled and the configuration shown in figure. Since inverse model of the plant is in off line

trained model, it tacks robustness.

DIRECT ADAPTIVE CONTROL (DAC)

In the direct adaptive control. The network is trained on line. And the weights of

connections are updated during each sampling interval. In this case’ the cost function is the plant

output error rather than the networks output error. The configuration of DAC is shown in figure.

The limitation of using this configuration is that one must have the some knowledge

about the plant dynamics i.e. Jacobin matrix of the plant. To solve the problems; initially, Psaltis

D.et al proposed a technique for determining the partial derivatives of the plant at its operating

point Xianzhang et al and Yao Zhang et al presented a simple approach, in which by using the

sign of the plant Jacobin. The modifications of the weights are carried out.

INDIRECT ADAPTIVE CONTROL

Narendra K S et al. Presented an indirect adaptive control strategy.

In this approach, two neutral networks for controller purpose and another for plant modeling and

is called plant emulator decides the performance of the controller. The configuration of indirect

adaptive control scheme becomes as shown FIG.3.

In direct learning the neutral controller, it is well known that the partial derivatives of the

controlled plant output with respect to the plant input (plant Jacobin) is required. One method to

overcome this problem is the use NN to identify the plant, and to calculate its Jacobin. Since the

plant emilator learning converges before the neutral controllers learning begins, an effective

neutral control system is achieved.

INTERNAL MODEL CONTROL (IMC)

The IMC uses two neutral networks for implementation. I n this configrurations,one

neutral networks is placed in parallel with the plant and other neutral network in series the plant.

The structure of nonlinear IMC is shown in FIG.4.

The IMC provides a direct method for the design of nonlinear feedback controllers. If a

good model of the plant is savable, the close loop system gives an exact set point tracking despite

immeasurable disturbance acting on the plant.

For the development of NN based IMC, the following two steps are required:

Plant identification

Plant inverse model

Plant identification is carried out using the forward modeling techniques. Once the network is

trained, it represents the perfect dynamics of the plant the error signal used to adjust the networks

weights is the difference between the plant output and the model output.

The neutral networks used to represent the inverse of the plant (NCC) are trained using

the plant itself. The error signal used to train the plant inverse model is the difference between

the desired plant and model outputs.

DIRECT NEURAL NETWORK MODEL REFERENCE ADAPTIVE

CONTROL:

The neural network approximates a wide variety of nonlinear control laws by adjusting

the weights in training to achieve the desired approximate accuracy. One possible MRAC

structure based on neural network is shown

In this configuration, control systems attempted to make the plant output YP(t) to follow

the reference model output asymptotically. The error signal

Used to train the neural network controller is the difference between the model and the plant

outputs, principally; this network works like the direct adaptive neural control system.

APPLICATION’S:

Aerospace

– High performance aircraft autopilots, flight path simulations, aircraft control

systems, autopilot enhancements, aircraft component simulations, aircraft

component fault detectors

Automotive

– Automobile automatic guidance systems, warranty activity analyzers

Banking

– Check and other document readers, credit application evaluators

Defense

– Weapon steering, target tracking, object discrimination, facial recognition, new

kinds of sensors, sonar, radar and image signal processing including data

compression, feature extraction and noise suppression, signal/image

identification

Electronics

– Code sequence prediction, integrated circuit chip layout, process control, chip

failure analysis, machine vision, voice synthesis, nonlinear modeling

Financial

– Real estate appraisal, loan advisor, mortgage screening, corporate bond rating,

credit line use analysis, portfolio trading program, corporate financial analysis,

currency price prediction

Manufacturing

– Manufacturing process control, product design and analysis, process and

machine diagnosis, real-time particle identification, visual quality inspection

systems, beer testing, welding quality analysis, paper quality prediction,

computer chip quality analysis, analysis of grinding operations, chemical product

design analysis, machine maintenance analysis, project bidding, planning and

management, dynamic modeling of chemical process systems

Medical

– Breast cancer cell analysis, EEG and ECG analysis, prosthesis design,

optimization of transplant times, hospital expense reduction, hospital quality

improvement, emergency room test advisement

Robotics

– Trajectory control, forklift robot, manipulator controllers, vision systems

Speech

– Speech recognition, speech compression, vowel classification, text to speech

synthesis

Securities

– Market analysis, automatic bond rating, stock trading advisory systems

Telecommunications

– Image and data compression, automated information services, real-time

translation of spoken language, customer payment processing systems

Transportation

– Truck brake diagnosis systems, vehicle scheduling, routing systems

CONCLUSION

Artificial neural networks thus have paved the way for automatic analysis of

biological data. The simultaneous analysis of millions of genes at a time has driven the new

field of computational biology—Genome informatics.

Application of artificial neural networks in Genome informatics has great

significance in this arena. The artificial neural network attains its knowledge through the

process of learning, which is similar to its inspiration---the Human brain.

But like all clouds with a silver lining Genomic engineering can be misused too, which can be a real threat to the mankind. Let us hope that man puts his brain for the welfare of his fellow beings and not against them.