interactive virtual environments - university of ottawapetriu/unimi2008-part4-model-3d... ·...

Interactive Virtual Environments

Modelling 3D Geometric and Elastic Properties

Emil M. Petriu, Dr. Eng., FIEEEProfessor, School of Information Technology and Engineering

University of Ottawa, Ottawa, ON, Canadahttp://www.site.uottawa.ca/~petriu

May 2008

Interactive Model-Based Hapto-Visual Teleoperation - a human operator equipped with hapticHCI can telemanipulate physical objects with the help of a robotic equipped with haptic sensors.

……

Tactile Sensor Interface

HapticRobot

InterfaceROBOT(k)

OBJ(i)

Robot Arm Controller

CyberGrasp ™ CyberTouch ™

Haptic Human I nterfaceUSER (k)

NETWORK NETWORK

…

{ [ 3D(j) & F(k,j) ], t }

AVATAR HAND ( k )

. . .

. . .

OBJ (N) OBJ (1)

3D Geometric & Elastic

Composite Model of Object{( x

p, y

p, z

p,E

p) | p = 1,..,P }

OBJ ( j )

Virtual Operation TheaterVirtual Operation Theater

Composite Haptic Interaction Vector between User (k) and Object (j)

ApplicationSpecific

InteractiveAction

Scenario

…

……


HapticRobot

InterfaceROBOT(k)

OBJ(i)



HapticRobot

InterfaceROBOT(k)

OBJ(i) OBJ(i)






NETWORK NETWORK

…

{ [ 3D(j) & F(k,j) ], t }

AVATAR HAND ( k )

. . .

. . .

OBJ (N) OBJ (1)



p, y

p, z

p,E

p) | p = 1,..,P }

OBJ ( j )



ApplicationSpecific

InteractiveAction

Scenario

{ [ 3D(j) & F(k,j) ], t }

AVATAR HAND ( k )

. . .

. . .

OBJ (N) OBJ (1)OBJ (1)



p, y

p, z

p,E

p) | p = 1,..,P }

OBJ ( j )



p, y

p, z

p,E

p) | p = 1,..,P }

OBJ ( j )

3D Geometric & Elastic3D Geometric & Elastic


p, y

p, z

p,E

p) | p = 1,..,P }{( x

p, y

p, z

p,E

p) | p = 1,..,P }

OBJ ( j )



ApplicationSpecific

InteractiveAction

Scenario

…

Model-based approach, based on the kinematics and dynamics of the object handled with the fingertips, provides a convenient representation of the dexterous manipulation. Quoting Salisbury et al.’s recent survey of haptic rendering, “improved accuracy and richness in object modeling and haptic rendering will require advances in our understanding of how to represent and render psychophysically and cognitively germane attributes of objects, as well as algorithms and perhaps specialty hardware (such as haptic or physics engines) to perform real-time computations”.----------------K. Salisbury, F. Conti, F. Barbagli, “Haptic Rendering: Introductory Concepts, ”IEEE Computer Graphics and Applications, Vol. 24, No. 2, pp. 24 – 32, 2004.

Modelling allows to simulate the behavior of a system for a varietyof initial conditions,excitations and systems configurations

The quality and the degree of the approximation of the model can be determined only by a validation against experimental measurements.

The convenience of the model means that it is capable of performing extensive parametric studies, in which independent parameters describing the model can be varied over a specified range in order to gain a global understanding of the response.

E.M. Petriu, "Neural Networks for Measurement and Instrumentation in Virtual Environments,” in Neural Networks for Instrumentation, Measurement and Related Industrial Applications , S. Ablameyko, L. Goras, M. Gori, V. Piuri - Eds.), NATO Science Series, Series III: Computer and System Sciences Vol. 185, pp.273-290, IOS Press, 2003

Discreet vs. Continuous Modelling of Physical Objects and Processes

DISCREET MODEL•sampling => INTERPOLATION COST

y(j) = y(A) + [ x(j)-x(B)] . [ y(B)-x(A)] / [x(A)-x(B)]

x

y

x(j)

y(j) = ? A

B

x

y

CONTINUOUS MODEL• NO sampling =>

NO INTEPPOLATION COST

Analog Computer vs. Neural Network Modelling of Continuous Physical Objects and Processes

Both the Analog Computers and the Neural Networks are continuous modelling devices.

The Analog Computer (AC) allows to solve the linear or nonlinear differential and/or integral equations representing mathematical model of a given physical process. The coefficients of these equations must be exactly known as they are used to program/adjust the coefficient-potentiometers of the AC’s computing-elements (OpAmps). The AC doesn’t follow a sequential computation, all its computing elements perform simultaneously and continuously. An interesting note, “because of the difficulties inherent in analog differentiation the [differential] equation is rearranged so that it can solved by integration rather than differentiation.” [A.S. Jackson, Analog Computation, McGraw-Hill Book Co., 1960].

The Neural Network (NN) doesn’t require a prior mathematical model. A learning algorithm is used to adjust, sequentially by trial and error during the learning phase, the synaptic-weights/coefficient-potentiometers of the neurons/computing-elements.

Similarly to the analog computer, a NN doesn’t follow a sequential computation algorithm, all its neurons performing simultaneously and continuously. The neurons are also integrative-type processing elements.

Compare the performance of three NN architectures used for 3D object shape modelling:

• Multilayer Feedforward (MLFF ) • Self-Organizing Map (SOM ) • Neural Gas Network

A.-M. Cretu, E.M. Petriu, G.G. Patry, “Neural-Network-Based Models of 3-D Objects for Virtualized Reality: A Comparative Study,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 1, pp.99-111, 2006.

NN Modelling of 3D Object Shape

The surface of an object is described by a set of zeros in the output response of the NN, while the interior or exterior regions of the object are described by negative or positive value, respectively.

Initial 3D pointcloudof sample

points representing the object

{( xi, yi, zi) | i = 1,...,N}

Neural Network model

of the object

{( xp, yp, zp) | p = 1,...,P}

Starting from a pointcloud of sample points capturing the shape of the object to be modeled, the NN produces a volumetric representation of the object.

The NN model provides a continuous representationof the object surface and permits an extensive study not only of the modeled object surface, but of theentire object volume. Thus it inherits the advantages of the polygonal and surface models, without inheriting their drawbacks.

It can be used to perform simple operations such as • object morphing, • set operations, and • object collision detection

Initial 3D pointcloud of samplepoints representing the object

{( xi, yi, zi) | i = 1,..., N}

xi, yi, zi xp, yp, zp

Transformed (translated, rotated,scaled, bent, tapered, twisted)

object{( Xi, Yi, Zi) | i = 1,..., N}

Neural Network Architecturefor 3D Object Representation

MLFF SOM Neural Gas

Xi, Yi, Zi

MLFF

Neural-Network modelof the object

{( xp, yp, zp) | p = 1,..., P}

Transformation Module:translation, rotation, scaling, and deformations (bending, tapering, twisting)

MLFF Neural Network

Pointcloud of sample pointsrepresenting the objectO{( xi, yi, zi) | i = 1,..., N}

xi, yi zi

Transformed object pointcloud{(X i, Yi, Z i) | i =1,..., N }

Xi, Yi ,Zi

The transformation module implements: * rotation, * translation, * scaling, and * deformations such as:

- bending, - tapering, and - twisting.

Transformation Module - Generation Mode

Original

Tapering

Rotation

Translation,Rotation,Scaling

Twisting

Bending

The transformation module has no hidden layer and the outputs have linear activation functions.

Knowing the equations, the network weights will be set such thatthe desired transformation is performed

The translation, rotation, scaling, and tapering will be encoded in the weights implementing the generalized 3D transformation matrix.

When provided with a set of points from a given reference model as inputs and those of a transformed model as targets, the NNtransformation module can learn and store in its weights information on how these points have been transformed and/or deformed.

Rotation and Translation

X

Z

Y

x

y

z

m1

m2

m3

n1n2

n3

q1

q2

q3t z

t y

t x

ψϕψϕ

ϕψθψϕθψθψϕθ

ϕθψθψϕθψθψϕθ

ϕθ

coscossincos

sin)sincoscossin(sin)coscossinsin(sin

cossin)sinsincossin(cos)cossinsinsin(cos

coscos

33

32

31

23

22

21

13

12

11

aqaqaqanananamamam

==

−=−=−=

=+=−=

=

Rotation

19080 points, 10-5-1, 5 extra surfaces, d=0.055, 1200 epochs, 2.8 hrs.

MLFF Representation - Results

250 points, 6-3-1, 1 extra surface, d=0.055, 550 epochs, 7 min.

7440 points, 8-4-1, 5 extra surfaces, d=0.055, 1100 epochs, 1 hr

51096 points, 20-10-1, 5 extra surfaces, d=0.055, 2000 epochs, 5.2 hrs.

2500 points, 12-6-1, 2 extra surfaces, d=0.06, 1020 epochs, 45 min.

MLFF Representation - Results19000 points, 14-7-1, 4 extra surfaces, d=0.055, 1100 epochs, 3.3 hrs

W12(i)

(x, y, z) points

X

Z

YReference

weight matrixW1

linear interpolation(n steps,i =1,..., n)betweenW1 andW2

X

Z

YTarget

weight matrixW2

O1

O2

(xm, ym, zm)of the morphedobjecti

x

z

y

MLFF Representation – Applications Object Morphing

MLFF Representation – Applications Set Operations

O1<0ANDO2<0

(x, y, z) belongs to intersection

O1<0ANDO2>0

(x, y, z) belongs to difference

X

Z

YNN

object1

O1

X

Z

YNN

object2

O2

( x, y, z )

O1<0ORO2<0

(x, y, z) belongs to union

97%

0%

2.3%

X

Z

YNN

object2

O

( x1, y1, z1 )( x2, y2, z2 )...( xn, yn, zn )

sampled pointsobject 1

Oi < 0

collision betweenobject 1 andobject 2

i=1,.., n

MLFF Representation – Applications Object Collision Detection

96.56%

MLFF Representation – Applications Object Recognition

W

alignmentinformation

RepresentationModule

X

Z

Y TransformationModule

points of thereference modeliin the database

sampledalignedpointsof transformedobject

Degree ofbelonging to a referencemodel i

points on thetransformedobject

The model-based recognitionis done by first aligning the transformed given object with the reference model stored the database.

The NN of the transformationmodule is trained using the reference model points as training points and the points of the given object as targets.In this way, the transformation information is stored in the NN weights.

After the alignment, a set of points is sampled from the transformed given object and then tested to see the degree of belonging to the reference model.

MLFF Representation – Applications Object Recognition

2.4% 4.9% 1.6% 99.1%

93.3% 3.3% 91.7% 3.1%

95% 99.1% 5.78% 98.3%

91.7% 6.6% 1.6% 92.5%

• simple and compact architecture• continuous volumetric model (though trained with

surface)• information about the entire object space• provides desired accuracy• represents objects of varied complexity• preserves details• morphing, set operations, recognition, collision

detection

• computationally intensive (for both learning and rendering)

• lack of local control of the object

MLFF Neural Network Modelling – Summary

Advantages Disadvantages

SOM and Neural Gas - Compressed Representation Models

Compressed NN model of the 3D object

{( xp, yp, zp) | p = 1,2, …, P }, where P<N

SOM / Neural GasPointcloud of sample pointsrepresenting the object O{( xi, yi, zi) | i = 1,..., N}

xi, yi zi

xp, yp zp

SOM Representation – NN Architecture

• Activation Function– soft competition

• Learning– unsupervised

input layer ...

[ x i, yi , zi ]

wji

yj K=2

K=1

winningneuron

A SOM has a input layer, where vector containing the x,y,z coordinates of points will be presented. Computations are feedforward in the first layer and bidirectional in the connection (output layer). The purpose of the lateral connection (represented by a Gaussian) is to reinforce the activation values of strong neurons and decrease the activation of weak ones. The output neurons are arranged on a grid, usually 2D. The network is based on competition - soft competition. The winning neurons is the one closest to theinput vector. Its value and the value of a given neighbourhood are updated during learning. After learning, two vectors belonging to the same cluster will be projected on two close neurons in the output space.

• Activation Functions:– soft competition– neighbourhood

ranking• Learning

– unsupervised

Neural Gas Representation – NN Architecture

The NG releases the neurons from the output grid. There are no connections between units in the connection layer. The nodes move independently over the data space. It also exploits the idea of soft competition, but the neurons to be updated are not selected according to topological relation, but rather according to rank. The rank is the rank the neurons have in an ordered list of distances between their weights and the input vector. NG converges quickly and to a lower distortion error.

Initial pointcloud

Neural Gas

SOM

19080 points 14914 points 13759 points

1125 points, 42 min.

1125 points, 26 min.

875 points,11 min.

875 points,24.5 min.

875 points22 min.

875 points, 10 min.

er= 0.0098

er= 0.0125

SOM and Neural Gas Modelling - Results

SOM and Neural Gas Modelling – Applications Object Morphing

SOM and Neural Gas Modelling – Applications Segmentation

• simple and compact (weights)• compressed• less memory usage• desired accuracy• objects of varied complexity• details • morphing, motion detection,

segmentation

• computational expensive for high accuracy

• no information about the object space

• no direct surface representation

SOM and Neural Gas Modelling – Summary

Advantages Disadvantages

MLF

FNe

ural

Gas

SOM

3.3 hrs 1 hrs

42 min.

26 min.

9 min.

3 min.

MLFF, SOM, and Natural Gas Modelling – Performance Comparison Training Time

0

50

100

150

200

250

300

350

Co

nst

ruct

ion

tim

e (

min

)

hand pliers face statue

Models

MLFFNN SOM Neural Gas

MLFNNcomputational time = construction time + generation time+rendering

SOM and Neural Gascomputational time =construction time + rendering

MLFF, SOM, and Natural Gas Modelling – Performance Comparison

As it can be seen, the MLFFNN is the most computational expensive, followed by NG. This time is only the construction time. However, the MLFFNN requires a generation time as well to reconstruct the model given the weights and self-organizing architectures require the rendering time. For the point based rendering technique, the self-organizing architectures are still less computationally expensive than the MLFFNN.

0

200

400

600

800

1000

1200

Sto

rage s

pace

(K

b)

hand face pliers statue

Models

Point CloudMLFFNNSOMNeural Gas

MLFF, SOM, and Natural Gas Modelling – Performance Comparison Compactness

The use of neural network modeling is advantageous from the point of view simplicity and compactness.

MLFNN – provide continuous models, information on the entire object space, convenient for many applications, however they are time consuming.

SOM and Neural Gas – provide compressed models while maintaining the properties of the objects, have very good accuracy, and they are less time consuming

The use of any specific techniques depends on the application requirements.

MLFF, SOM, and Natural Gas Modelling of 3D Objects

CONCLUSIONS

Neural Network Adaptive Sampling of 3D Object Elastic Properties

A.-M. Cretu, E.M. Petriu, “Neural-Network –Based Adaptive Sampling of Three-Dimensional-Object Surface Elastic Properties,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 2, pp. 483-492, 2006.

Recovery of the elastic material properties requires touching each point of

interest on the explored object surface and then conducting a strain-stress

relation measurement on each point.

< = ≤≤ ⋅=

0

maxmax

max

pppp

ppppp

ififE

εεσσεεεσ

The elastic behaviour at any given

point (xp, yp, zp) on the object surface

is described by the Hooke’s law:

where Ep is the modulus of elasticity ,

s p is the stress, and e p is the strain

on the normal direction.

Tactile probing is a time consuming Sequential operation

Find fast sampling procedures able to minimize the number of the sampling points by selecting only those points that are relevant to the elastic characteristics.

non-uniform adaptive sampling algorithm of the object’s surface,

which exploits the SOM (self-organizing map) ability to find optimal finite quantization of the input space.

Initial 3D geometricmodel of the object's surface

{( xi, yi, zi) | i = 1,...,N}SOM / Neural Gas

Adaptive-sampled3D geometricmodel of the object surface{( xp, yp, zp) | p = 1,...,P}

RoboticTactileProbing

Adaptive-sampled3D geometric&

elastic composite modelof object's surface

{( xp, yp, zp, Ep) | p = 1,...,P}

xi, yi, zi

xp, yp, zp

Ep

Adaptive Sampling Control of the Robotic Tactile Probing of Elastic Properties of 3D Object Surfaces

SOM (Self Organizing Map) and Neural Gas NN architectures are both used to build compressed model of the 3D object originally defined as a point-cloud.

The weight vector will consist of the 3D coordinates of the object’s points.

During the learning procedure, the model will contract asymptotically towards the points in the input space, respecting their density and thus taking the shape of the object encoded in the point-cloud.

Data point-clouds obtained with a range scanner are used to train the network. Normalization is employed to remove redundant information from a data set, by a linear rescaling of the input vectors such that their variance is 1.

In order to evaluate the quality of the models, a straightforward measure of the precision is used. The precision is estimated as the average distance between each data vector and its winning neuron .

a) b) c)

d) e) f)

g) h) i)

(a) Training data setof 3721points

(b) Neural Gas network, error=0.0112,

(c) SOM, error=0.0133(d) Noisy data set,

random 0 – 0.1,e) Neural Gas network

error=0.0383, (f) SOM, error=0.0266(g) Noisy data set,

random 0 – 0.04, (h) Neural Gas network

error=0.0224, (i) SOM, error=0.0241

Robustness to noisy training data

Training Neural Gas networkwith a map size of 25×45 for 20 epochs, for α0 = 0.5, λ0 = number of neurons/2 and a SOM, with the initial neighborhood radius σ0 =5, and a map size of 25×45, trained for 100 epochs, with data corrupted by different levels of noise.

The initial set of 3721 points is reduced to 1125 points.

It takes approximately 250s for the SOM to build a model of a sphere, while it takes approximately double for the Neural Gas NN. However, even for a larger number of training epochs (5 times more) the SOM does not reach the same accuracy as the Neural Gas NN does, for data that is not very noisy (a random noise level below 0.1).

SOM suffers from the boundary problem. The models obtained look as if they contain cavities.

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0 0 - 0.04 0 - 0.05 0 - 0.1 0 - 0.2 0 - 1

random noise level

rela

tive

erro

r

Neural Gas SOM

For low levels of noise the Neural Gas network performs better than SOM. For higher level of noise, SOM tends to smooth the effect of noise, while the Neural Gas network, which has high sensitivity,follows the noisy patterns.

• The first column presents three views of the originalpoint-cloud of 19,080 points representing a human face.

• The second column presents the compressed model of 1,152 points obtained using The Neural Gas network.

• The third column presents the compressed model of 1,152 points obtained using SOM.

Qualitative comparison between the Neural Gas and the SOM adaptive sampled models.

• The map sizes are equal for both networks.

• The first column represents the original point-cloud,

• The second column represents the Neural Gas model.

• The third column represents the SOM model.

For the 14,914 points of the original point-cloud model given in the first figure,it takes 24 min. to build the Neural Gas model shown in the second figure and 11 min. to build the SOM model shown in the third figure (for the same map size of 25x35 in both cases).

00.20.40.60.8

11.21.41.61.8

2

5 10 20 50 100 1000

number of training epochs

rela

tive

erro

r

Neural Gas SOM

For both, Neural Gas and SOM, networks the quality is improving with the number of training epochs

On the whole the quality of the Neural Gas models appears to be better. Because of the boundary problem, the SOM models are to be avoided for non-noisy data.

Neural Gas and SOM neural networks are both able to compress the initial model with the desired degree of accuracy.

The number of points can be further reduced by reducing the map size. However, there is a compromise to be made between the quality of the resulting compressed model and the map size.

Neural Gas networks are able to model an entire scene of objects while the SOM networks are not able of such a performance.

Starting from a 3D point-cloud, a neural gas NN yields a reduced set of points on the 3D object’s surface which are relevant for the tactile probing. The density of these tactile probing points is higher in the regions with more pronounced variations in the geometric shape. A feedforward NN is then employed to model the force/displacement behavior of selected sampled points that are probed simultaneously by a force/torque sensor and theactive range finder.

3D pointcloudof data

Samplepoints

Deformationprofiles

ForceMeasurements

FeedforwardNeural Network

F

profile(f0)profile(f1)profile(f2)profile(f3)

f0f1f2f3

Neural gasnetwork

Rangefinder

Force/Torquesensor

Neural Network Mapping an Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications

Sampling points selected withthe neural gas network.

Variable elasticity object usedfor experimentation.

(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006 ).

Sampling points selected withthe neural gas network for the ball.

Elastic ball usedfor experimentation.


Different magnitudes of a normal force are applied successively on the selected sampling points using the probe attached on the force/torque sensor and a range profile is collected with the laser range finder for each force magnitude.


There is no need to recover the explicit displacement information from the range profiles. Instead the NN models use the raw range data as a function of applied force, F, without explicitly defining values for the displacement. For each cluster of similar elasticity, a feed-forward NN with two input neurons (F and a), 45 hidden neurons (H1-H45) and one output neuron (Z), is used to learn the relation between forces and the corresponding geometric profiles provided by the range finder.

F

Z...

H1

H45

H2

α

The NN associated with each material were trained for 10,000 epochs using the Levenberg-Marquardt variation backpropagationalgorithm with the learning rate set to 0.1. The whole data set is used for training in order to provide enough samples. The training takes approximately 10 min. on a Pentium IV 1.3GHz machine with 512MB memory. For the rubber, the sum-squared error reached during training is 3.7 x10-3, for cardboard is 3.5 x10-2 while for the foam is 2.2x10-2. As expected, the error is lower for the rubber where data is more compact and less noisy, while it remains slightly higher for the cardboard and even higher for the foam. But in all cases, excellent convergence is achieved.

(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).


Deformation profiles for semi-stiff material (cardboard).


Deformation profiles for smooth material (foam).

Real and modeled deformation curves using neural network for semi-stiff material (cardboard) under a normal force of: a) F=0.1N, b) F=0.37N, and c) F=2.65N.

(a) (b) (c)

Real and modeled deformation curves using neural network for smooth material (foam) under a normal force of: a) F=0N, b) F=0.93N, and c) F=3.37N.

(a) (b) (c)


Real and modeled deformationcurves using neural network for rubber under a normal force of: a) F=0N, b) F=65.52N, and c) F=80.5N.

(a) (b)

(c)


(a) (b)

Real and modeled deformation curves using neural network for rubber under forces applied at different angles: a) F=65N, a1=10° and F=65N, a2=170°,b) F=36N, a1=25°, and F=36N, a2=155

(from .A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).


Real, modeled and estimated deformation profiles detail of estimated deformation profiles using neural network for rubber ball for increasing forces applied at 75-degree angle.

Hardware Neural NetworkArchitectures

ANNs / Neurocomputers =>architectures optimized for neuron model implementation

general-purpose, able to emulate a wide range of NN models;special-purpose, dedicated to a specific NN model.

Hardware NNs consisting of a collection of simple neuron circuits provide the massive computational parallelism allowing for a higher Model rendering speed.

ANN VLSI Architectures:• analog ==> compact,high speed,

asynchronous, no quantizationerrors, convenient weight “+”and “X”;

• digital ==> more efifcient VLSI technology,robust, convenient weight storage;

Pulse Data Representation:• Pulse Amplitude Modulation (PAM) -not satisfactory for NN processing;

• Pulse Width Modulation (PWM);• Pulse Frequency Modulation (PFM).

Number of nodes

0

103

106

109

1012

103 106 109 1012 Node complexity[VLSI area/node]

RAMsSpecial-purpose neurocomputers

General-purpose neurocomputersSystolic arrays

Computational arays

Conventional parallel computers

Sequential computers

[from P. Treleaven, M. Pacheco, M. Vellasco, “VLSI Architectures for Neural Networks,”IEEE Micro, Dec. 1989, pp. 8-27]

Pulse Stream ANNs: combination of different pulse data representation methodsand opportunistic use of both analog and digital implementation techniques.

HARDWARE NEURAL NETWORK ARCHITECTURES USING RANDOM-PULSE DATA REPRESENTATION

Looking for a model to prove that algebraic operations with analog variables can be performed by logical gates, von Neuman advanced in 1956 the idea of representing analog variables by the mean rate of random-pulse streams [J. von Neuman, “Probabilistic logics and the synthesis of reliable organisms from unreliable components,” in Automata Studies, (C.E. Shannon, Ed.), Princeton, NJ, Princeton University Press, 1956].

The “random-pulse machine” concept, [S.T. Ribeiro, “Random-pulse machines,” IEEE Trans. Electron. Comp., vol. EC-16, no. 3, pp. 261-276,1967], a.k.a. "noise computer“, "stochastic computing“, “dithering”deals with analog variables represented by the mean rate of random-pulse streams allowing to use digital circuits to perform arithmetic operations. This concept presents a good tradeoff between the electronic circuit complexity and the computational accuracy. The resulting neural network architecture has a high packing density and is well suited for very large scale integration (VLSI).

Interactive telemanipulation applications require real-time rendering of complex NN models

HARDWARE ANN USING MULTI-BIT RANDOM-DATA REPRESENTATION

Generalized b-bit analog/random-data conversion and its quantization characteristics

VRV

RVRQ

CLOCKCLK

VRP

b-BITQUANTIZER

X XQ

ANALOG RANDOMSIGNALGENERATOR

-D/2 0R

p(R)1/D

+D/2

++

.(k+0.5) D(k-0.5) D.

XQ

X

k

k+1

k-1

0

b D.

1/Dp.d.f.of VR

D/2D/2

b D. (1- b) D.

.V= (k- b) Dk D.

E.M. Petriu, L. Zhao, S.R. Das, V.Z. Groza, A. Cornell, “Instrumentation Applications of Multibit Random-Data Representation,” IEEE Trans. Instrum. Meas., Vol. 52, No. 1, pp. 175- 181, 2003.

32 266 5003.2

1

1.2

x2is

x2ditis

x2RQis4

2

dZis

dHis

dLis

MAVx2RQis

is

Moving Average ‘Random Pulse -to- Digital ” Conversion

2-bit random-data NN architectureof an auto-associative memory

P1, t1 P2, t2 P3, t3

Training set

30

P30x1

30x30

n

30x1

a

30x1W

)*hardlim( PWa =

Recovery of 30% occluded patterns

Model-based approach, based on the kinematics and dynamics of the object handled with the fingertips, provides a convenient representation of the dexterous manipulation. “Improved accuracy and richness in object modeling and haptic rendering will require advances in our understanding of how to represent and render psychophysically and cognitively germane attributes of objects, as well as algorithms and perhaps specialty hardware (such as haptic or physics engines) to perform real-time computations” [K. Salisbury, F. Conti, F. Barbagli, “Haptic Rendering: Introductory Concepts,” IEEE Computer Graphics and Applications, Vol. 24, No. 2, pp. 24 – 32, 2004].

Neural Networks which are able to learn nonlinear behaviors from a limited set of measurement data can provide efficient and compact multi-media object modeling solutions. Due to their continuous, analog-like, memory behavior, NNs are able to provide instantaneously an estimation of the output value for input values that were not part of the initial training set.

NNs consisting of a collection of simple neuron circuits provide the massive computational parallelism offering efficient storage, model transformation, and real-time rendering capabilities for large numbers of composite geometric & haptic object models involved in the model-based interactive telemanipulation.

Conclusions

S. Andrews, “Measurement-Based Modeling of Contact Forces and Textures for Haptic Rendering,” M.Sc. Thesis, 2007.

J.-C. Delannoy, “Modeling Articulated Bodies with Distributed Mass for Virtual Interactive Environments,” M.A.Sc. Thesis, 2006.

A.-M. Cretu, “Neural Network Modeling of 3D Objects for Virtualized Reality Applications,” M.A.Sc. Thesis, 2003.

A. Moica, "Coprocessor for Decoding Multi-Valued Pseudo-Random Sequence Patterns," M.A.Sc. Thesis, 2000.

L. Zhao, "Random Pulse Artificial Neural Network Architecture," M.A.Sc. Thesis, 1998.

S.S.K. Yeung, “Model-Based Tactile Object Recognition Using Pseudo-Random Encoding“, Ph.D. Thesis, 1996.

C. Archibald, “A Computational Model for Skills-Oriented Robot Programming,“ Ph.D. Thesis, 1995.

D.M. Colven, "Tactile Pattern Recognition Using Neural Networks," M.A.Sc. Thesis, 1993.

Ottawa “U” Research Group – Relevant Graduate Theses

J. Zhou, N.D. Georganas, E.M. Petriu, X. Shen, F. Marlic, “Modelling Contact Forces for 3D Interactive Peg-in-Hole Virtual Reality Operations,” Proc. I²MTC 2008 – IEEE Int. Instrum. Meas. Technol. Conf., pp. 1397-1402, Victoria, BC, Canada, May 2008.

E.M. Petriu, A.-M. Cretu, P. Payeur, “Neural Network Modeling Techniques for the Real-Time Rendering of the Geometry and Elasticity of 3D Objects,” Proc. SOFA 2007, 2nd IEEE Int. Workshop on Soft Computing Applications, pp. 11-16, Gyula , Hungary, and Oradea. Romania, Aug. 2007.

A.-M. Cretu, E.M. Petriu, “Neural-Network –Based Adaptive Sampling of Three Dimensional-Object Surface Elastic Properties,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 2, pp. 483 - 492, 2006.

A.-M. Cretu, E.M. Petriu, G.G. Patry, “Neural-Network-Based Models of 3-D Objects for Virtualized Reality: A Comparative Study,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 1, pp.99-111, 2006.

A.-M. Cretu, E.M. Petriu, P. Payeur, “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” Proc. IST 2006, IEEE Intl. Workshop on Imagining Systems and Techniques, Minori, Italy, pp.17–22, April 2006.

A.-M. Cretu, J. Lang, E.M. Petriu, “A Composite Neural Gas-Elman Network that Captures Real-World Elastic Behavior of 3D Objects,” Proc. IMTC/2006, IEEE Instrum. Meas. Technol. Conf., pp. 1063-1068, Sorrento, Italy, April 2006.

E.M. Petriu, S.K.S. Yeung, S.R. Das, A.-M. Cretu, H.J.W. Spoelder, “Robotic Tactile Recognition of Pseudorandom Encoded Objects,” IEEE Trans. Instrum. Meas., Vol. 53, No. 5, pp. 1425-1432, 2004.

Ottawa “U” Research Group - Publications in Modelling3D Object Geometry and Elastic Properties

interactive virtual environments - university of ottawapetriu/unimi2008-part4-model-3d... ·...

Documents