introduction to the theory, numerical methods and applications of ill-posed problems ... ·...
TRANSCRIPT
1
Introduction to the theory, numerical methods and
applications of ill-posed problems
(lecture course)
Anatoly Yagola
Professor, Dr. Sc., Department of Mathematics, Faculty of
Physics, Moscow State University, Moscow 119991,
Russia, e-mail: [email protected]
Orebro, May 2-4 , 2016
CONTENTS
Introduction
Well-posed and ill-posed problems
Elements of functional analysis
Examples of ill-posed problems
Definition of the regularizing algorithm
Ill-posed problems on compact sets
2
CONTENTS
Extremal problems statement
Solvability of an extremal problem.
Simplest necessary and sufficient
conditions of minimum
Convex functionals
Solvability of a convex programming
problem
3
CONTENTS
Criteria of convexity and strong
convexity
Least square method. Pseudoinversion
Minimizing sequences
Some methods for solving one-
dimensional extremal problems
Method of steepest descent
4
CONTENTS
Conjugate gradient method
Newton’s method
Zero-order methods
Conditional gradient method
Projection of conjugate gradients
method
5
CONTENTS
Numerical methods for solving ill-posed
problems on special compact sets
Ill-posed problems with a source-wise
represented solution
Tikhonov’s regularizing algorithm
Generalized discrepancy principle
6
CONTENTS
Incompatible ill-posed problems
Numerical methods for the solution of
Fredholm integral equations of the first
kind
Convolution type equations
7
Introduction
In the lecture course we will describe
fundamentals of the theory of ill-posed
problems so as numerical methods for their
solution if different a priori information is
available. For simplicity, only linear
equations in normed spaces are considered.
8
Introduction
Although, it is clear that all similar
definitions can be introduced also for
nonlinear problems in more general metric
(even also topological) spaces.
Numerous inverse problems can be found in
different branches of physics (astrophysics,
geophysics, spectroscopy, nuclear physics,
etc.).
9
Introduction
Mostly, these inverse problems are ill-
posed. It means that small deviations of
input data (due to experimental errors) can
produce large errors in an approximate
solution.
10
Introduction
The modern theory of ill-posed problems
started seventy years ago (1943) when
famous Russian mathematician Andrey
Tikhonov published in Soviet Mathematics
Doklady magazine a paper “On stability of
inverse problems”.
11
Introduction
In 1963 Tikhonov formulated the definition
of a regularizing algorithm. He and two
other Russian mathematicians Mikhail
Lavrentiev and Valentin Ivanov became
founders of the modern theory of inverse
and ill-posed problems.
12
Introduction
13
Introduction
Now this theory is greatly developed
throughout the world. The most eminent
specialists in Sweden are Lars Elden from
Linkoping and Larisa Beilina from
Gothenburg.
14
Course literature
A.N.Tikhonov, A.V.Goncharsky,
V.V.Stepanov, A.G.Yagola. Numerical
methods for the solution of ill-posed
problems. - Kluwer Academic Publishers,
1995.
A.N.Tikhonov, A.S.Leonov, A.G.Yagola.
Nonlinear ill-posed problems. V. 1, 2 -
Chapman and Hall, 1998.
15
Course literature
L. Beilina, M.V. Klibanov. Approximate
global convergence and adaptivity for
coefficient inverse problems. – Springer,
2012.
16
Recommended reference literature
O.M. Alifanov, E.A. Artioukhine, S.V.
Rumyantsev. Extreme Methods for Solving
Ill-Posed Problems with Applications to
Inverse Heat Transfer Problems. – Begell
House Inc., 1995.
A. Bakushinsky, A. Goncharsky. Ill-posed
problems: Theory and applications. -
Kluwer Academic Publishers, 1994.
17
Recommended reference literature
A.B. Bakushinsky, M.Yu. Kokurin.
Iterative Methods for Approximate Solution
of Inverse Problems. – Springer, 2005.
H.W. Engl, M. Hanke, A. Neubauer.
Regularization of Inverse Problems. –
Kluwer Academic Publishers, 1996.
18
Recommended reference literature
V.K. Ivanov, V.V. Vasin, V.P. Tanana.
Theory of Linear Ill-Posed Problems and its
Applications. – VSP, 2002.
M.M. Lavrentiev, L.Ya. Saveliev. Operator
Theory and Ill-Posed Problems. – De
Gruyter, 2006.
V.V. Vasin, A.L. Ageev. Ill-Posed
Problems with A Priori Information. – VSP,
1995.19
Well-posed and ill-posed problems
Let us consider an operator equation:
Az=u,
where is a linear operator acting from a
Hilbert space Z into a Hilbert space U. It is
required to find a solution of the operator
equation z corresponding to a given
inhomogeneity (or right-hand side) u.
20
Well-posed and ill-posed problems
The problem of solving the operator
equation is called to be well-posed
(according to Hadamard) if the following
three conditions are fulfilled:
21
Well-posed and ill-posed problems
Usually, a choice of the space of solutions
(including a choice of the norm) is
determined by requirements of an applied
problem. A mathematical problem can be
ill-posed or well-posed depending on a
choice of a norm in a functional space.
22
Well-posed and ill-posed problems
Lavrentiev’s example
Condition 3) is fulfilled if
||z||A = ||Az||,
A is an injective operator.
23
Elements of functional analysis
Linear spaces
Metric spaces
Normed spaces
Banach spaces
Euclidean spaces
Hilbert spaces
24
Elements of functional analysis
Example. A finite-dimensional vector space
Rn that is very well known from Linear
algebra.
25
Elements of functional analysis
Example. A space C[a,b] of (real)
functions defined and continuous on a
segment [a,b] with the norm:
||y||C[a,b]=max{|y(s)|, s[a,b]}.
Convergence in this space is called uniform
convergence.
26
Elements of functional analysis
Definition. A sequence xnN, n=1, 2, …; is
called fundamental sequence if for any 0
there exists a natural number K such that for
any nK and any natural number p ||xn+p-
xn||.
If any fundamental sequence converges then
the normed space is called the complete
space (or Banach space).
27
Elements of functional analysis
Example. Rn and C[a,b] are Banach
spaces.
28
Elements of functional analysis
29
Elements of functional analysis
Definition. A linear space is called the Euclidean
space if for any two elements x, yE is defined a
real number (x,y) (a scalar product) such that:
1) for any x, yE (x,y)=(y,x);
2) for any x1,x2, yE (x1+x2,y)=( x1,y)+(x2,y);
3) for any x, yE and any real (x,y)=
(x,y);
4) for any xE (x,x)0, and (x,x)=0 if and
only if x=0.
30
Elements of functional analysis
Example. A finite-dimensional vector space
Rn is an Euclidean space with a scalar
product (x,y)=x1y1+…+xnyn, where x1,…,xn;
y1,…,yn; are components of vectors x and y
respectively.
31
Elements of functional analysis
32
Elements of functional analysis
33
Elements of functional analysis
34
Elements of functional analysis
Linear operators
35
Elements of functional analysis
36
Elements of functional analysis
37
Elements of functional analysis
Definition. A sequence yn,, n=1,2,…, of
elements of a normed space N is called
compact if any its subsequence has a
convergent subsequence. If space N is not
Banach space, then it has a fundamental
subsequence.
38
Elements of functional analysis
Definition. A linear operator А: N N is
called compact if for any bounded
sequence i=1, 2,…; elements of the
sequence zn=Ayn of elements in is
compact.
Definition. A linear compact operator is
completely continuous.
39
Elements of functional analysis
Theorem. Any completely continuous
operator is bounded (consequently,
continuous).
In finite-dimensional spaces any linear
operator is completely continuous. It is not
true in infinite-dimensional spaces.
40
Elements of functional analysis
Example. Let us consider the identity
operator I: , Iy=y for any y. It is bounded
but not compact.
Theorem. Let А be the Fredholm integral
operator mapping from L2[a,b] into L2[a,b].
Then А is completely continuous.
41
Examples of ill-posed problems
Bb
42
The integral operator A is completely continuous (compact and continuous) while
acting from ],[2 baL into ],[2 dcL
Examples of ill-posed problems
Let us suppose that the inverse operator is
continuous. It is very easy to arrive to a
contradiction. If A is an injective operator
then an inverse operator exists. Evidently, if
an operator B is continuous and an operator
A is completely continuous then the
operator BA is completely continuous.
43
Examples of ill-posed problems
44
If inverse to A operator is bounded then
any bounded sequence is compact! It is not
true in infinite-dimensional spaces.
Examples of ill-posed problems
Infinite-dimensional range of a completely
continuous operator R(A) is not closed! It is
easy to prove using Banach’s Theorem. If
the injective linear operator A is
continuously mapping an infinite-
dimensional Banach space on an infinite-
dimensional Banach space then the inverse
operator is contnuous.
45
Examples of ill-posed problems
46
Definition of the regularizing
algorithm
47
Definition of the regularizing
algorithm
48
Definition of the regularizing
algorithm
A problem of solving an operator equation
is called to be regularizable if there exists at
least one regularizing algorithm. Directly
from the definition it follows that if there
exists one regularizing algorithm then
number of them is infinite.
49
Definition of the regularizing
algorithm
At the present time, all mathematical
problems can be divided into following
classes:
well-posed problems;
ill-posed regularizable problems;
ill-posed nonregularizable problems.
50
Definition of the regularizing
algorithm
Not all ill-posed problems are regularizable,
and it depends on a choice of spaces Z, U.
Russian mathematician L.D. Menikhes
constructed an example of an integral
operator with a continuous closed kernel
acting from C[0,1] into L2[0,1] such that an
inverse problem is nonregularizable.
51
Definition of the regularizing
algorithm
52
Definition of the regularizing
algorithm
53
Definition of the regularizing
algorithm
54
Definition of the regularizing
algorithm
55
Definition of the regularizing
algorithm
The next very important property of ill-
posed problems is impossibility of error
estimation for a solution even if an error of
a right-hand side of an operator equation or
an error of an argument in a problem of
calculating values of an operator is known.
This basic result was also obtained by A.B.
Bakushinsky for solving operator equations.
56
Definition of the regularizing
algorithm
57
Definition of the regularizing
algorithm
From the definition of the regularizing
algorithm it follows immediately if one
exists then there is infinite number of them.
While solving ill-posed problems, it is
impossible to choose a regularizing
algorithm that finds an approximate solution
with the minimal error.
58
Definition of the regularizing
algorithm
It is impossible also to compare different
regularizing algorithms according to errors
of approximate solutions. Only including a
priori information in a statement of the
problem can give such a possibility, but in
this case a reformulated problem is well-
posed in fact. We will consider examples
below.
59
Definition of the regularizing
algorithm
Regularizing algorithms for operator
equations in infinite dimensional Banach
spaces could not be compared also
according to convergence rates of
approximate solutions to an exact solution
as errors of input data tend to zero. The
author of this principal result is V.A.
Vinokurov.
60
61
Consider the results obtained by Vinokurov.
Let be a linear continuous injective operator
acting in Banach space and the inverse operator
. be unbounded on . Suppose that
. is an arbitrary positive function such that
. as , and is an arbitrary method
to solve the problem.
The following equality holds for elements except
maybe for a first category set in :
A uniform error estimate can only exist on a first
category subset in .
AZ
1A 1AD
R
z
Z
0 0
zR ,,suplim
0
Z
Definition of the regularizing
algorithm
62
Definition of the regularizing
algorithm
63
Ill-posed problems on compact
sets
64
Ill-posed problems on compact
sets
65
Ill-posed problems on compact
sets
Tikhonov (1943) “On stability of inverse
problems” (Soviet Mathematics Doklady)
66
Ill-posed problems on compact
sets
67
Ill-posed problems on compact
sets
68
Ill-posed problems on compact
sets
69
Ill-posed problems on compact
sets
70
Extremal problems statement
71
Extremal problems statement
72
Solvability of an extremal
problem
73
Solvability of an extremal
problem
74
Solvability of an extremal
problem
75
Solvability of an extremal
problem
76
Solvability of an extremal
problem
77
Solvability of an extremal
problem
78
Convex functionals
79
Convex functionals
80
Convex functionals
A classification of extremal problems:
81
Convex functionals
82
Convex functionals
83
Convex functionals
84
Convex functionals
Theorem. If (*) is a convex programming
problem then any point of a local minimum
is a point of the global minimum.
Convex functionals have no local minima.
85
Convex functionals
86
Solvability of a convex
programming problem
Theorem. A convex continuous (lower
semicontinuous) functional f(x) on a closed
bounded convex set of a Hilbert space H
has a minimum point (i. e. a convex
programming problem (*) is solvable).
Theorem (Weierstrass). A weakly lower
semicontinuous functional on a weak
compact has a minimum point.
87
Solvability of a convex
programming problem
We need to prove two assertions only:
1. A convex continuous (lower
semicontinuous) functional is weakly lower
semicontinuous.
2. A closed bounded convex set of a
Hilbert space is a weak compact.
The theorem is true for reflexive Banach
spaces only.88
Solvability of a convex
programming problem
89
Solvability of a convex
programming problem
90
Solvability of a convex
programming problem
91
Criteria of convexity and
strong convexity
92
Criteria of convexity and
strong convexity
93
Least square method.
Pseudoinversion
94
Least square method.
Pseudoinversion
95
System of normal equations
Least square method.
Pseudoinversion
96
Least square method.
Pseudoinversion
97
Least square method.
Pseudoinversion
98
Pseudoinverse to A operator
Least square method.
Pseudoinversion
99
Minimizing sequences
100
Minimizing sequences
101
Minimizing sequences
102
Minimizing sequences
103
Minimizing sequences
104
Minimizing sequences
105
Minimizing sequences
106
Some methods for solving one-
dimensional extremal problems
107
Some methods for solving one-
dimensional extremal problems
108
Method of steepest descent
109
Method of steepest descent
110
Method of steepest descent
111
Conjugate gradient method
112
Conjugate gradient method
113
Conjugate gradient method
114
Conjugate gradient method
115
Conjugate gradient method
116
Conjugate gradient method
117
Conjugate gradient method
118
Conjugate gradient method
119
Newton’s method
120
Newton’s method
121
Newton’s method
122
Newton’s method
123
Zero-order methods
Coordinate descent.
Hooke-Jeeves method.
Nelder-Mead method.
Random search.
Genetic algorithms.
124
Conditional gradient method
125
Conditional gradient method
126
Conditional gradient method
127
Projection of conjugate
gradients method
128
Projection of conjugate
gradients method
129
Projection of conjugate
gradients method
130
Projection of conjugate
gradients method
131
Projection of conjugate
gradients method
132
Projection of conjugate
gradients method
If we have linear constraints of inequality
type, the method can be applied also. It is a
bit more complicated.
133
Numerical methods for solving ill-posed
problems on special compact sets
134
Numerical methods for solving ill-posed
problems on special compact sets
135
Numerical methods for solving ill-posed
problems on special compact sets
136
Numerical methods for solving ill-posed
problems on special compact sets
137
Bounded monotonic functions
Let is a set of monotonic nonincreasing on
[a, b] functions is .
Lemma. is a compact in Lp[a,b], p>1.
The Lemma guarantees convergence of
quasisolutions (or η-quasisolutions) to the
exact solution in Lp[a,b], p>1.
138
Bounded monotonic functions
Theorem (Goncharsky, Yagola). If the exact solition of the
operator equation is a nonincreasing bounded monotonic
continuous function then quasisolutions (or η-
quasisolutions) converge to the exact solution uniformly
on any segment . If the exact solution is
piecewise continuous then the convergence is uniform on
any segment that doesn’t include the points of
discontinuity of the exact solution (so called piecewise
uniform convergence).
139
Numerical methods for solving ill-posed
problems on special compact sets
140
Numerical methods for solving ill-posed
problems on special compact sets
141
Numerical methods for solving ill-posed
problems on special compact sets
142
Numerical methods for solving ill-posed
problems on special compact sets
143
Numerical methods for solving ill-posed
problems on special compact sets
144
Numerical methods for solving ill-posed
problems on special compact sets
145
Numerical methods for solving ill-posed
problems on special compact sets
146
Numerical methods for solving ill-posed
problems on special compact sets
147
148
Ill-posed problems with a source-wise
represented solution
(1)
is a linear injective operator.
Assume the next a priori information: is
sourcewise represented with a linear compact
operator :
(3)
Here is a reflexive Banach space.
Suppose is injective, is known exactly, .
uzA
UZA :
z
ZVB :
vBz
V
B A uu
149
Set and define the set
Minimize the discrepancy on .
If , then the solution is
found. Denote . Otherwise, we change
to and reiterate the process.
If is found, then we define the approximate
solution of (1) as an arbitrary solution of the
inequality
1n
nvVvBvzZzZn ,,:
uAzzF
nZzuAz :min
nZ
n
1n
nn
n
nz
nZzuAz
150
Theorem 1: The process described above converges: . . There exists (generally speaking, depending on ) such that for . Approximate solutions strongly converge to . as .
Proof The ball is a bounded closed set in . The set is a compact in for any , since is a compact operator. Due to Weierstrass theorem the continuous functional attains its exact lower bound on .
Clearly, , where
. is the integer part of a number.
n 00
z 0 nn 0,0
nz
z 0
nvVvVn :
VnZ Z n
B zF
nZ
NZvBz
otherwise1
integerpositiveais
v
vvN
151
Therefore is a finite number and there is such that for any . The inequality for any is evident. Thus, for all the approximate solutions . belong to the compact set , and the method coincides with the quasisolutions method for all sufficiently small positive . The convergence follows from the general theory of ill-posed problems.
Remark 1: The method is a variant of the method of extending compacts (Ivanov, Dombrobskaya).
n0
0 nn 0,0 Nn 0
0,0
nz 0nZ
zzn
152
Theorem 2: For the method described above there exists an a posteriori error estimate. It means that a functional exists such that as and at least for all sufficiently small positive .
Remark 2: The existence of the a posteriori error estimation follows from the following. If by . we denote the space of sourcewise represented with the operator solutions of (1), then . Since is a compact set, then . is a -compact space.
,u 0, u
0 ,uzzn
ZZ
B
1n nZZ nZ
Z
153
An a posteriori error estimate is not an error estimate
in general meaning that is impossible in principle
for ill-posed problems. But it becomes an upper
error estimate of the approximate solution for
“small” errors , where depends on the
exact solution .0 0
z
154
155
The operators and are known with errors. Let
there be linear operators , such that
. , . Denote the vector of errors
by . For any integer define a
compact set .
Find a minimal positive integer number such
that the inequality
has a nonempty set of solutions.
Then the a posteriori error estimation is
A B
AhABhB
Ah hAAA
Bh hBBB
BA hh ,, n nvVvvBzZzZ
BB hhn ,,:,
nn
nhhAhBhuzA BAhBhAh ABA
}
,:max{,,, ,
nhhAhBhuzA
ZzzznhBAu
BAhBhAh
hnnBhh
ABA
BBA
156
A posteriori error estimation
For some ill-posed problems it is possible to find a
so-called a posteriori error estimation.
Let be an exact injective operator with closed
graph and be a -compact space.
Introduce a function such that . , , , :
The function is an a posteriori error
estimation for the problem (1), if as
A
Z ,u Zz
0 z ],0( z Uu uu
,, uuRz
,u
0, u
0
157
Tikhonov’s regularizing algorithm
Let be Hilbert spaces, be a closed
convex set of a priori constraints such that ,
. , be linear operators. On a set
introduce the Tikhonov's functional:
where is a regularization parameter.
(2)
For any , and bounded linear operator
. the problem (2) is solvable and has a unique
solution .
UZ , ZD
D0
A hA ,,uAh
22zuzAzM h
0
DzzM :inf
Uu 0
hA
Dz
158
A priori choice of
A regularizing algorithm using the extreme
problem (2) for : to construct
such that as .
If is an injective operator, and ,
. as , then as ,
i.e., there is the a priori choice of .
zM zz
0
A Dz 0
0
2
h0 zz
0
159
Generalized discrepancy principle
(a posteriori choice of )The incompatibility measure of (1) on :
Let it can be computed with an error , i.e.,
instead of there is such that
The generalized discrepancy:
The generalized discrepancy is continuous and
monotonically non-decreasing for .
D
DzuzAAu hh :inf,
0
hAu , hAu ,
hhh AuAuAu ,,,
222
, hh AuzhuzA
0
160
The generalized discrepancy principle to choose the
regularization parameter:
1) If the condition is not just,
then is an approximate solution of (1);
2) If the condition is just,
then the generalized discrepancy has a positive
zero and .
If is an injective operator, then .
Otherwise, , where is the normal
solution of (1), i.e., .
222, hAuu
0z
222, hAuu
**
zz
A zz
0lim
*
0lim zz
*z
uAzDzzz ,:inf*
161
If are bounded linear operators, is a closed
convex set, , , the generalized
discrepancy principle are equivalent to the
generalized discrepancy method:
find
inf
hAA, D
D0 Dz
222,,: hh AuzhuzADzz
162
Inverse problem for the heat conduction
equation
There is a function , we want to
find such that
as .
We can write that
0,
0,0
,0,02
tlw
tw
Tltxwaw xxt
lLTwu ,0, 2
lWxwxz ,00, 12 xzxz
0
l l
dxx
xzxzxzduu
0 0
22222
,
163
The problem may be written in the form of integral
equation
where is the Green function:
The problem is solved for the parameters
. , the function is taken
such that .
l
dxxzTxGu0
,,
txG ,,
1
2
2 expsinsin,,n
lt
l
na
l
nx
l
ntxG
0.1,1.0,0.1 lTa u
u 05.0
164
The exact solution ( ) and the approximate
solution ( ).
)(xz xz
165
The Euler equation
The Tikhonov's functional is a strongly convex functional in a Hilbert space.
The necessary and sufficient condition for to be a minimum point of on a set of a prioriconstraints is
If is an interior point of , then , or
We obtain the Euler equation.
zM
z
zM D
DzzzzM
0,
z D 0
zM
uAzzAA hhh
**
166
Error-free methods
As the first example we consider so-called the “L-
curve method” (P.C. Hansen). In this method the
regularization parameter in Tikhonov functional
is selected as a point maximum curvature of the L-
curve {(ln||Ahz - u||, ln||z||): 0}.
But this method cannot be used for the solution of
ill-posed problems because the L-curve doesn’t
depend on h and (see the theorem). Everybody
can easily prove that this method is inapplicable to
solving the simplest finite-dimensional well-posed
problems (e.g., equation z=1).
167
Another very popular “error free” method is GCV – the
generalized cross-validation method (G. Wahba), where
(Ah, u) is found as the point of the global minimum
of the function
G() = ||(AhAh* + I)-1u|| [tr(AhAh
* + I)-1]-1, 0.
This method is not applicable for the solution of ill-
posed problems including ill-posed systems of linear
algebraic equations (see the theorem above). It is
possible construct well-posed systems of linear
algebraic equations the GCV method failed for their
solution. Let Z = U = R2,
21
11,
1
2Au
168
Here h 0. Very easy to calculate the GCV solution
zgcv and prove that it converges to (-1/3, -1/3)* instead
of ze = (-3, 1)* when h 0.
A lot of other examples could be found in a paper
by
V. Titarenko and A. Yagola (2000) Vestnik
Moskovskogo Universiteta, ser. 3. Fizika.
Astronomia (4), 15 (in Russian).
169
Publications on error-free methods
A.S.Leonov and A.G.Yagola. Can an ill-posed
problem be solved if the data error is unknown? -
Moscow University Physics Bulletin, v.50, N 4, 1995,
pp.25-28.
C.R. Vogel. Non-convergence of the L-curve
regularization parameter selection method. - Inverse
Problems, 1996, v.12, pp. 535-547.
A.G.Yagola, A.S.Leonov, V.N.Titarenko. Data errors
and an error estimation for ill-posed problems. –
Inverse Problems in Engineering, 2002, v. 10, N 2,
pp. 117-129.
Incompatible ill-posed problems
170
171
172
173
174
175
176
177
Numerical methods for the solution of
Fredholm integral equations of the first
kind
178
179
180
181
Finite-dimensional approximation of Tikhonov’s functional
182
Convolution type equations
183
184
185
186
187
188
189
190
191
192
193
194
195
Examples and Applications
196
Functions convex along lines parallel to
coordinate axes
Consider an n-dimensional Euclidean space Rn, n < .
A set Rn is convex along all lines parallel to coordinate axes if i [1,n] x1,x2 such that
x1=(a1,…, ai-1, x1i, ai+1,…, an),
x2=(a1,…, ai-1, x2i, ai+1,…, an)
and (0,1): x3= x1 +(1-) x2.
197
A cross is an example of a set convex along
coordinate axes
198
A function z(x) on is convex downwards
along all lines parallel to an i-th coordinate
axis if x1,x2 such that
x1=(a1,…, ai-1, x1i, ai+1,…, an),
x2=(a1,…, ai-1, x2i, ai+1,…, an)
and (0,1):
z( x1 +(1-) x2 ) z(x1) + (1-) z(x2)
199
Let n* [0,n]. Consider functions z(x) given on .
By Mn*n ( ) define the set of functions z(x) that
are convex downwards along all lines parallel to
n* first coordinate axes and convex upwards along
all lines parallel to (n-n*) last coordinate axes.
Assume there exist finite numbers CL and CU such
that x and z(x) Mn*n( ): CL z(x) CU.
200
201
202
Theorem 1.: Let there be a sequence {zm} and an element z such that m1,…,+: zm Mn
n* (), z Lp(), p>1, ||zm-z||Lp() 0 as m + , where is an open bounded set. Then from the sequence {zm} a subsequence {zm
(k)} may be taken that converges to a function žMn
n* () at any point of and ž=z in Lp( ).
Corollary 1.: Mnn* ( ) is a compact set in Lp( ).
Corollary 2.:The sequence {zm(x)} considered in
Theorem 2.1 converges to the function ž(x) at any point of .
203
Theorem 2.: Let ||zm - z||Lp() 0 as m,
where zm , zMnn* (), p 1 and is an
open bounded set. Then the sequence {zm }
converges to z uniformly on any closed set
.
204
Let D=[a1,b1] [a2,b2] … [an, bn]. On each segment [ai ,bi], we define a grid Xi={xi
j}j=1ni such
that ai = xi1 < xi
2 <…< xini = bi. Let X=X1 X2 …
Xn. A vector of indices J=(j1, j2,…, jn) for a grid point with coordinates (x1
j1,x2j2,…,xn
jn). Then the point is written as xJ.
For any xD there is a set BJ=[x1j1, x1
j1+1]… [xn
jn, xnjn+1]: x BJ. As an approximation of a
function z(x) we use a function zN (x) that is linear on grid values of z(x) at vertices of BJ.
205
After finite dimensional approximation we
obtain a set ŽM which is a polytope.
If x1, x2, x3 are grid points that belong to a line
parallel to an i-th coordinate axis and there
is no another grid point between them, then
for a uniform grid Xi: -z1+ 2z2 - z3 0 (in*)
or z1 - 2z2 + z3 0 (otherwise). (zk = z(xk))
206
Error estimation
1) Find the minimum and the maximum values foreach coordinate of ŽM
. Denote them by zli and
zui, 1 i n. They form vectors žl, žu.
2) Secondly, using žl, žu we construct functions zl(x)and zu(x) close to ZM
such that zZM: zl(x)
z(x) zu(x).
Therefore, we should minimize a linear functionon a convex set. We may approximate the set bya convex polyhedron and solve a linearprogramming problem. The simplex-method orthe method to cut convex polyhedra may be used.We also may construct the sequence W0 W1… Wm of convex polyhedrons contained thepoint of minimum.
207
Let D=[0,d1 ] [0,d2 ], d1, d2<+, and for w(x,y,t) there are the heat conduction equation and zero boundary conditions:
Denote z(x,y)=w(x,y,0), u(x,y)=w(x,y,T), 0<T<+ . Therefore
u(x,y)=G(x, y,,, T) z(,) dd.
0),,(0),0,(
0),,(0),,0(
2
1
2
2
2
22
tdxwtxw
tydwtyw
y
w
x
wa
t
w
208
Assume the exact solution zM20 (D). We set n1 =
n2 = 11, d1 = d2 = 1.0, the grids are uniform, a = 1.0, T=0.001. As the exact solution the function z(x,y) = sin( x) · sin( y) is taken. The approximate right-hand side we take as u = ū. The error of finite dimensional approximation = 0.01· ||ū|| 0.005.
In the figure there is an upper function zU(x,y) that bounds all approximate solutions. To construct it we use additional grid values.
We find that ||zU-zL|| = 0.212 ( 0.424 · ||z||).
209
Linear ill-posed problems on sets
of convex functions on two-dimensional sets
This case is more complicated. It is possible to prove that
the set of bounded convex 2D functions is a compact in
Lp( ). A set Rn is closed bounded set. So we can
find a quasisolution and its error estimation.
The detailed description of the algorithm so as the
algorithm of the previous paragraph can be found in our
joint publications with Valery Titarenko.
210
211
1
3
54 h
0
z
2
Fig.1
Electron microscopy
212
z
z'
1
0
2
Fig.2
213
отδ
IAφ , (1)
where
Aφ= ρr
zρdρφγρK0
z)()z,( ,
>0 – error of assignment of the right part of the equation
(1), i.е. δII relrel
, rel
IφA , (0)III rel - relative
intensity.
214
2
2
AL
IφφFδ
, (2)
At that it is enough to find such an element δ
φ , that
2δφFδ .
γ
γ
ρzотjρziij
N
j
N
ihIhφKφf
1 1
2
)( (3)
At finite-difference approximation set Z transforms
into set
ρzi
ρziii
iii
Niφ
NNi
Ni
φφφ
φφφ
φZinfl
infl
,1,2,0,
1,1,
1,2,
0,2
0,2
:ˆ11
11
,
(4)
215
Let )( jT , mj ,0,1, ( ρzNm ) – apexes of a convex
limited polyhedron Z .
Lemma. Let Zφ . Then the unique representation is correct
m
j
jjTaφ
1
)( ,
at that m,,,j,aj
210 .
216
It is obvious, that mm TT
RZ and ZR 1 ,
Where mR - set of vectors mm RR , that have all non-
negative coordinates mξ R , if mj,ξj
,1,2,0 .
Let us examine function )()( ξTfξY , determined on
set mR .
We need to find such an element mδξ R , that
2)( δξYδ . The approximate solution of the original
problem is found then by the formula δδ
Tξφ .
Let us examine operator T from mR in mR ,
determined by the formula
m
j
mjj
Rξ,TξξT1
)(.
217
0,0 0,5 1,0
0,1
0,2
(
Z)
Z
218
0,00
0,05
0,10
0,15
0,20
0,25
0,0 0,5 0,9
exact
calculated
219
0,0 0,5 1,00,0
0,1
0,2
(
Z)
Z
220
INVERSE PROBLEM OF
CATHODOLUMINESCENCE
MICROTOMOGRAPHY
221
The Scheme of Installation
1. Focused electrical probe
2. Object under investigation
3. Region of generation of nonequilibrium carriers
4. Ellipsoidal mirror
5. Diaphragm with detector
222
Problem
Develop method for determination of optoelectrical local
properties of cathodoluminescence objects with resolution of
micrometer part, having at our disposal the set of measurements
of intensity values.
Describe the scheme of experiment, mathematical statement and
the method of solution of the problem, which is ill-posed.The interaction of focused electrical probe with
cathodoluminescence substance was modulated. An alternative
method of microtomography in cathodolumenscence mode is
presented. The solution is based on confocal ellipsoidal
mirror [Phang J.C.H, Chan D.C.H.].
The photon rays transport in luminescence volume of
specimen and ellipsoid are calculated.
223
We have to solve the next inverse problem:define the internal quantum yield of the material
from Fredholm integral equation of the first kind:
],0[),( 0Rss
mirrorthetorespectin
objecttheofdeflectionthexdsssxKxIR
0
01 ,)(),()(
where - intensity, measured in experiment, as function of deflection
of the object in vertical direction,
-the distance from the surface of the object,
-maximal depth of penetration of electrons into the object,
-some continuous function, which was calculated by numerical
methods (the physical sense of is that is the contribution
into the total intensity the layer with center on the depth s and thickness
ds).
)(xI
0R
),(1 sxK
dssxK ),(1
s
],0[)( 0Rs ],[)( maxmin2 xxLxI
224
A Priori Information
Let it is known that the solution of the problem is sourcewiserepresented with help of completely continuous integral operator:
0
0
002 ],0[,)(),()(
R
RsdsKs
otherwise
sssK
,0
,2,2/2/1)cos((),(2
where
We shall consider that:
)(0 s ],0[ 02 RL ],0[)( 020 RLs
For solving the problem under such a priori information the method of extending compacts, which was described above, is used.
225
Model Calculations Results
226
227
An Inverse Problem of Nuclear Physics
228
An inverse problem of nuclear physics
Experiment:
Fig.1: 1 - the target for producing bremsstrahlung beam, 2 - the sample under consideration, D –
detector.
Passing through the first target the accelerated electrons produce the bremsstrahlung
beam (γ-rays). The bremsstrahlung spectrum is continuous. The sample 2 is bombarded
by the γ-rays. The scattered γ-rays are detected.
γe−
γ
1 2
D
229
Experimental data processing
Nuclear reaction:
Constraints:
A priori :
A posteriori:
is a monotone nondecreasing function
is a convex upwards function
is a monotone nonincreasing function
63 62
29 29Cu Cu n
0 ( ) 90, [10,24.1]E E
( ), [10,16]E E
( ), [16,18]E E
( ), [18,24.1]E E
230
Fig.2: (• • •) – the approximate cross section
from the Center of Data of Photonuclear
experiments (http://depni.sinp.msu.ru/cdfe/);
(• • •) – the approximate solution found by
Tikhonov regularization;
( – ) – the functions
bounded the set of approximate solutions
from below and from above .
( ), ( )low upper
E E
231
Experimental data processing
Nuclear reaction:
Constraints:
A priori:
A posteriori:
is a monotone nondecreasing function
is a convex upwards function
is a monotone nonincreasing function
is a convex downwards function
is a monotone nondecreasing function
is a convex upwards function
is a monotone nonincreasing function
34 33
16 15S P p
0 ( ) 45, [12.3,25.3]E E
( ), [12.3,16]E E
( ), [16,17]E E
( ), [17,18.5]E E
( ), [18.5,20]E E
( ), [20,22]E E
( ), [22,23]E E
( ), [23,25.3]E E
232
Fig.3: (• • •) – the approximate cross section
from the Center of Data of Photonuclear
experiments;
(• • •) – the approximate cross section found
by Tikhonov regularization;
( ─ ) – the functions
bounded the set of approximate solutions
from below and from above .
( ), ( )low upper
E E
233
Image reconstruction for gravitational lens
The system QSO 2237+0305, known as the
“Einstein Cross”: 4 quasar images against
the background of the lensing galaxy.
Several observation were carried out using
the Huble Space Telescope and Nordic
Optical Telescope.
Model of Kernel
ResidualsStar Kernel
PSF (Kernel) profile
Approximation of the star from
the frame with 2-dimensional
Gauss profile
FWHM ~ 5 pixels
Tikhonov Regularization
• Ill-posed problem
• Smoothing function:
)(*][2
zzkzMU
u
• Regularization parameter from discrepancy principle:
0,*
Uuzk
}:][inf{][ ZzzMzM
• Solution : z
A priori information
True Image = Galaxy + Quasar Components
K
kkbykaxkIyxgyxz
1
),(),(),(
K=4 , number of quasar components
K=5 , number of quasar components + galaxy nuclear
A priori information
• Galaxy model 2
model)(
Gggg
• Nonnegativity of the solution, zij 0
• Galaxy: assumption about smoothness
}exp{01
modeln
en r/r b)I((r)g
bn=2n-0.324 for 1n4
generalized de Vaucouleurs profile (Sersic’s model)
; 2
)(G
gg BVWLG ,21,2
A priori information
Sourcewise representation:
'*]'[ zrzRz
rsk *
PSF * PSF PSF FinalSourceTotal
),(),(),(1
yxgcybxrayxzK
kkkk
Results: L2
2
2)(
Lsersicggg
Observed image Reconstructed image
Results: L2
Galaxy Error distribution
Results: W21
Observed image Deconvolved image
2
21)(
Wsersicggg
Results: W21
Galaxy Error distribution
Results: MCS
Observed image Deconvolved image
2
2)(
Lgrgg
rs *kernel
Results: MCS
Quasar components Galaxy
Results: MCS
Quasar components Error distribution
Results: TV
11
1
12
1,1,,11,1
)(
N
m
N
nnm
gnm
gnm
gnm
gg
Observed image Deconvolved image
Results: TV
Quasar components Galaxy
Using parallel computing for solving multidimensional ill-
posed problems
248
249
1. Introduction
J
jij
i
ij
ijijj
i
r
M
r
rrMB
135
0)(3
4
iB
?jM
The equation describing the magnetic field Bi induced by sources of magnetic fields
Mj, located at a distance rij from the sensor i, is defined as
The inverse problem is to identify the permanent magnetization M (both strengths
and directions) using measurements of the magnetic flux density B.
1.1. An example of a multi-dimensional ill-posed problem
250
1. Introduction
35
0 )()(
))(),((3
4)(
qr
qMqr
qr
qMqrrBq
The total field of the ship can be expressed by the
integral
In other words, the source of the magnetic field with
magnetic moment M, located at a point with radius
vector q, creates at point with radius vector q a
magnetic field with induction B:
1.1. An example of a multi-dimensional ill-posed problem
1. General mathematical formulation of the problem
q
V
dVqr
qMqr
qr
qMqrrB
35
0 )()(
))(),((3
4)(
But this statement of the problem is computationally
very difficult
1. Introduction1.1. An example of a multi-dimensional ill-posed problem
1. Some different types of simplifications of the numerical model for the assigned problem
But all these simplifications can be applied only in specific cases (not in general)251
1D-problem: Dividing the ship into subdivisions with
constant values of magnetization
2D-problem: Approximation the hull of the ship
by an ellipsoid
2D-problem: Approximation the hull of the ship
by polyhedrons
2D-problem: Approximation the hull of the ship
by a plane
252
1. Introduction
But if we do not have any a priori information about the investigated object we can
not use mentioned simplifications. In this case we have to solve the problem “in
general” that is very difficult to perform on common PC.
The answer is only one: we have to use parallel
computing
For example, how can 67 500 parameters be inverted
efficiently?
1.1. An example of a multi-dimensional ill-posed problem
3. Inverse problem without any simplifications
253
2. Using parallel computing
Parallel computing is a form of computation in which many calculations are carried
out simultaneously, operating on the principle that large problems can often be
divided into smaller ones, which are then solved concurrently (“in parallel”). Parallel
computation can be performs on multi-processor clusters or on multi-core
computers having multiple processing elements within a single machine.
But not every problem can be parallelized efficiently
Large problem
Sub problem 1
Su
b p
rob
lem
N
Solving of smaller
problems
Process 1
Process 2
Process 3
Process N
. . . .
. .
Result 1
Result 2
Result 3
Result N
. . . .
. .
Result of solving
of the large
problem
Result of intermediate
calculationsResult
2.1. The main idea of parallel computing
254
2. Using parallel computing
It will be shown that parallelizable fraction for multidimensional Fredholm integral
equation of the 1st kind is ~ 99,(9)% that gives us high effectiveness
2.2. Parallel computing limitations
The speed-up of a program as a result of
parallelization is observed as Amdahl’s
law
N
PP
S
)1(
1
S -- the speed-up of the program (as a factor
of its original sequential runtime)
N -- number of processors
P -- the fraction of the program that is
parallelizable.
255
3. Parallelization of multidimensional ill-posed problems
The total field of the ship expressed by the integral
can be replaced by an equivalent 3D integral equation
V
ssssss dvzyxMzyxzyxKzyxB ),,(),,,,,(),,(
3.1. Formulation of the problem
x
x
y
y
z
z
R
L
R
L
R
L
dxdydzzyxMzyxrtsKrtsB ),,(),,,,,(),,(
q
V
dVqr
qMqr
qr
qMqrrB
35
0 )()(
))(),((3
4)(
and then, after change of variables,
256
m
iii
m
iiiN
l
N
l
N
l
n
jjj
m
m
lll
nm
llljjjzyx
N
j
N
j
N
j
nm
iiijjj
n
rtszyx
m
iii
m
iiim
iii
M
MBMKhhhKhhhhhh
M
MFMFgrad
x y zs t r
321
321
1 2 3
321321321321
1 2 3
321321
321
321
321
1 1 1
3
11 1 1
3
1
2
)(
When we solve minimization problem by conjugate gradient method, it is necessary to
calculate values of the functional MF
MFgrad
and its gradient
Finite-difference approximation of the Tikhonov functional is
Finite-difference approximation of its gradient is
Structure of algorithm allows to divide the large problem into smaller ones which
are then solved “in parallel”.
s t r x y zN
j
N
j
N
j n
N
i
n
jjj
N
i
N
i
m
iii
nm
iiijjjzyx
m
rts MBMKhhhhhhMF1 1 1
3
1
2
1 1 13
3
11 2 3 1
321
2 3
32121321
3.2. Finite-difference approximation of the functional and its gradient
3. Parallelization of multidimensional ill-posed problems
257
Paralleling of the functional is consider on the example
rtszyx hhhhhh ,,,,, are skipped for simplicity.
2 rtszyx NNNNNN ,
steps and smoothing functional
22
1
32
1
2
13
3
222
3
1
22
1
32
1
2
13
3
212
3
1
22
1
22
1
2
13
2
111
3
1
22
1
12
1
2
13
1
111
3
1
2
1
2
1
2
1
3
1
22
1
2
1
2
13
3
1
1
321
2 3
32121
1
321
2 3
32121
1
321
2 3
32121
1
321
2 3
32121
1 2 3 1
321
2 3
32121321
i
jjj
i i
m
iii
m
iii
m
i
jjj
i i
m
iii
m
iii
m
i
jjj
i i
m
iii
m
iii
m
i
jjj
i i
m
iii
m
iii
m
j j j n i
n
jjj
i i
m
iii
nm
iiijjj
m
BMK
BMK
BMK
BMK
BMKMF
Process 1
Process 2
Process 18
Process 24
. . . .
Result 1
Result 2
Result 18
Result 24
Aggregate
result
. . . .
+
. . . .
++
. . . .
++
3. Parallelization of multidimensional ill-posed problems3.3. One possible scheme of paralleling I
258
The scheme of calculating the value of the functional for a) zero process, b) non-
zero processes.
rts NNNN
i = 1
s
i = i + 1
i = ”number of
processes”
s(i) = ”square
of sum”
i = i + ”quantity
of processes”- 1
a) b)
][][ MMF
Ni Ni
MM
M
sMFMF ][][
)(is
3. Parallelization of multidimensional ill-posed problems3.3. One possible scheme of paralleling II
259
zyx NNNN
The scheme of calculating value of the gradient of the functional for: a) zero
process; b) non-zero processes.
i = 1
grad = 0
s, k
grad(k) = s
i = i + 1
k = ”number of
process”
s(k) =
”gradient”
k =k + ”quantity
of processes”-1
M
a) b)
Ni Nk
)(ks
M
M
3. Parallelization of multidimensional ill-posed problems3.3. One possible scheme of paralleling III
260
Results of restoring 100x15x15x3=67500 magnetization parameters. Time of calculation is
~29 hours on 256 processors (Intel Xeon E5472 3.0 GHz)
3. Parallelization of multidimensional ill-posed problems3.4. Some examples of calculations
261
4. Conclusion
The proposed method can be efficiently applied for
solving multidimensional Fredholm integral equations
of the 1st kind in many areas of physics were it is
necessary to solve inverse problems such as:
• radiophysics
• optics
• acoustics
• spectroscopy
• geophysics
• tomography
• image processing
• etc.
The testing calculations were performed on the Computing Cluster of the Moscow State University.
The work was partially supported by RFBR grant 10-01-91150- NFSC.