Using Deep Neural Network for SolvingBackward Heat Equations

Farinaz Mostajeran

Joint Work with:Prof. S. M. Hosseini & Prof. R. Mokhtari

Sep. 2019Summer School in Graz

Thank You

.. Prof. Gundolf Haase for kind invitation and covering travel expenses

.. Prof. Gundolf Haase and Ms. Tanja Weiss for picking this great place,organizing, shuttle, accommodation

.. Participants

Thank You

.. Prof. Gundolf Haase for kind invitation and covering travel expenses

.. Prof. Gundolf Haase and Ms. Tanja Weiss for picking this great place,organizing, shuttle, accommodation

.. Participants

Thank You

.. Prof. Gundolf Haase for kind invitation and covering travel expenses

.. Prof. Gundolf Haase and Ms. Tanja Weiss for picking this great place,organizing, shuttle, accommodation

.. Participants

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network1. Inference Problems 2. Identification Problems

...4 Future Works

..2 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network1. Inference Problems 2. Identification Problems

...4 Future Works

..2 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network1. Inference Problems 2. Identification Problems

...4 Future Works

..2 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network1. Inference Problems 2. Identification Problems

...4 Future Works

..2 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network

...4 Future Works

..3 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionBackward Heat Equation (BHE)

Let u(x, t) satisfy the following heat conduction equation

ut(x, t) = κ(t) ∆u(x, t), (x, t) ∈ Ω× (0, T ),

under the final temperature condition

u(x, T ) = g(x), x ∈ Ω,

and the Dirichlet boundary condition

u(x, t) = h(x, t), x ∈ ∂Ω, t ∈ (0, T ),

where κ(t), g(x), h(x, t) are known functions, Ω is a connected andbounded domain and ∂Ω is the boundary of the domain Ω.

..4 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionBackward Heat Equation (BHE)

Let u(x, t) satisfy the following heat conduction equation

ut(x, t) = κ(t) ∆u(x, t), (x, t) ∈ Ω× (0, T ),

under the final temperature condition

u(x, T ) = g(x), x ∈ Ω,

and the Dirichlet boundary condition

u(x, t) = h(x, t), x ∈ ∂Ω, t ∈ (0, T ),

where κ(t), g(x), h(x, t) are known functions, Ω is a connected andbounded domain and ∂Ω is the boundary of the domain Ω.

..4 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionBackward Heat Equation (BHE)

Let u(x, t) satisfy the following heat conduction equation

ut(x, t) = κ(t) ∆u(x, t), (x, t) ∈ Ω× (0, T ),

under the final temperature condition

u(x, T ) = g(x), x ∈ Ω,

and the Dirichlet boundary condition

u(x, t) = h(x, t), x ∈ ∂Ω, t ∈ (0, T ),

where κ(t), g(x), h(x, t) are known functions, Ω is a connected andbounded domain and ∂Ω is the boundary of the domain Ω.

..4 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionBackward Heat Equation (BHE)

Let u(x, t) satisfy the following heat conduction equation

ut(x, t) = κ(t) ∆u(x, t), (x, t) ∈ Ω× (0, T ),

under the final temperature condition

u(x, T ) = g(x), x ∈ Ω,

and the Dirichlet boundary condition

u(x, t) = h(x, t), x ∈ ∂Ω, t ∈ (0, T ),

where κ(t), g(x), h(x, t) are known functions, Ω is a connected andbounded domain and ∂Ω is the boundary of the domain Ω.

..4 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionBackward Heat Equation (BHE)

Let u(x, t) satisfy the following heat conduction equation

ut(x, t) = κ(t) ∆u(x, t), (x, t) ∈ Ω× (0, T ),

under the final temperature condition

u(x, T ) = g(x), x ∈ Ω,

and the Dirichlet boundary condition

u(x, t) = h(x, t), x ∈ ∂Ω, t ∈ (0, T ),

where κ(t), g(x), h(x, t) are known functions, Ω is a connected andbounded domain and ∂Ω is the boundary of the domain Ω.

..4 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionBackward Heat Equation (BHE)

Let u(x, t) satisfy the following heat conduction equation

ut(x, t) = κ(t) ∆u(x, t), (x, t) ∈ Ω× (0, T ),

under the final temperature condition

u(x, T ) = g(x), x ∈ Ω,

and the Dirichlet boundary condition

u(x, t) = h(x, t), x ∈ ∂Ω, t ∈ (0, T ),

where κ(t), g(x), h(x, t) are known functions, Ω is a connected andbounded domain and ∂Ω is the boundary of the domain Ω.

..4 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionNumerical Methods for Solving BHEs

.. Boundary Element MethodH. Han, D.B. Ingham, Y. Yuan, The boundary element method for the solution of the backward heatconduction equation, Journal of Computational Physics, 116:292-9, 1995.

.. Finite Difference Method

.. Method of Fundamental Solutions

.. Tikhonov Regularization

.. Radial Basis Functions

.. Deep Neural Network

..5 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionNumerical Methods for Solving BHEs

.. Boundary Element Method

.. Finite Difference MethodK. Lijima, Numerical solution of backward heat conduction problems by a high order lattice-free finitedifference method, Journal of the Chinese Institute of Engineers, 27(4):61120, 2004.

.. Method of Fundamental Solutions

.. Tikhonov Regularization

.. Radial Basis Functions

.. Deep Neural Network

..5 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionNumerical Methods for Solving BHEs

.. Boundary Element Method

.. Finite Difference Method

.. Method of Fundamental SolutionsN. S. Mera,The method of fundamental solutions for the backward heat conduction problem, InverseProblems in Science and Engineering, 13(1):65-78, 2005.

.. Tikhonov Regularization

.. Radial Basis Functions

.. Deep Neural Network

..5 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionNumerical Methods for Solving BHEs

.. Boundary Element Method

.. Finite Difference Method

.. Method of Fundamental Solutions

.. Tikhonov RegularizationX. L. Feng, Z. Qian, CL. Fu, Numerical approximation of solution of nonhomogeneous backward heatconduction problemin bounded region, Mathematics and Computers in Simulation, 79:177-88, 2008.

.. Radial Basis Functions

.. Deep Neural Network

..5 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionNumerical Methods for Solving BHEs

.. Boundary Element Method

.. Finite Difference Method

.. Method of Fundamental Solutions

.. Tikhonov Regularization

.. Radial Basis FunctionsM. Li, T. Jiang, Y.C. Hon, A meshless method based on RBFs method for nonhomogeneous backward heatconduction problem, Engineering Analysis with Boundary Elements, 34:785792, 2010.

.. Deep Neural Network

..5 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionNumerical Methods for Solving BHEs

.. Boundary Element Method

.. Finite Difference Method

.. Method of Fundamental Solutions

.. Tikhonov Regularization

.. Radial Basis Functions

.. Deep Neural NetworkM. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics informed deep learning (Part I): data-driven solutionsof nonlinear partial diffeerential equations, arXiv:1711.10561v1, 2017.M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics informed deep learning (Part II): data-drivendiscovery of nonlinear partial diffeerential equations, arXiv:1711.10566v1, 2017.

..5 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionHistory

..6 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionHistory

P. Diaconis, Bayesian numerical analysis, in: Statistical Decision Theory and Related Topics IV, vol.1, 1988, pp.163-175.

..6 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionHistory

T. Graepel, Solving Noisy Linear Operator Equations by Gaussian Processes: Application to Ordinary and PartialDifferential Equations, ICML, 2003.S. Skinear operators and stochastic partial differential equations in Gaussian process regression, in: Artificial NeuralNetworks and Machine Learning, ICANN 2011, Springer, 2011, pp.151-158.

..6 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionHistory

P. Hennig, S. Hauberg, Probabilistic solutions to differential equations and their application to Riemannian statistics,in: AISTATS, 2014, pp.347-355.P. Hennig, M.A. Osborne, M. Girolami, Probabilistic numerics and uncertainty in computations, Proc. R. Soc. A 471(2015) 20150142.

..6 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionHistory

J. Cockayne, Probabilistic meshless methods for partial differential equations and Bayesian inverse problems, arXivpreprint, arXiv:1605.07811, 2016.I. Bilionis, Probabilistic solvers for partial differential equations, arXiv preprint, arXiv:1607.03526, 2016.

..6 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

IntroductionHistory

W. E, A proposal on machine learning via dynamical systems, Communications in Mathematics and Statistics, 5(1):1-11, 2017.Z. Long, Y. Lu, X. Ma, and B. Dong, PDE-NET: learning PDEs from data, arXiv:1710.09668v2, 2018.

..6 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network

...4 Future Works

..7 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksBiological Neural Networks

..8 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksBiological vs. Artifitial Neural Networks

..9 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksShallow Fully Connected Neural Networks (NN)

..10 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksDeep Fully Connected Neural Networks (DNN)

..11 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksFeed-forward Network Functions

zj =n∑

i=1

wijxi + bj , aj = σ(zj)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..12 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksFeed-forward Network Functions

zj =n∑

i=1

wijxi + bj , aj = σ(zj)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..12 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksFeed-forward Network Functions

zj =n∑

i=1

wijxi + bj , aj = σ(zj)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..12 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksCommonly Used Activation Functions

.. Linear functionσ(x) = c x

.. Sigmoid functionσ(x) = (1 + e−x)−1

.. Rectified Linear Unit (ReLU)functionσ(x) = max(0, x)

.. Sinusoid functionσ(x) = sin(x)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..13 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksCommonly Used Activation Functions

.. Linear functionσ(x) = c x

.. Sigmoid functionσ(x) = (1 + e−x)−1

.. Rectified Linear Unit (ReLU)functionσ(x) = max(0, x)

.. Sinusoid functionσ(x) = sin(x)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..13 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksCommonly Used Activation Functions

.. Linear functionσ(x) = c x

.. Sigmoid functionσ(x) = (1 + e−x)−1

.. Rectified Linear Unit (ReLU)functionσ(x) = max(0, x)

.. Sinusoid functionσ(x) = sin(x)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..13 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksCommonly Used Activation Functions

.. Linear functionσ(x) = c x

.. Sigmoid functionσ(x) = (1 + e−x)−1

.. Rectified Linear Unit (ReLU)functionσ(x) = max(0, x)

.. Sinusoid functionσ(x) = sin(x)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..13 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksCommonly Used Activation Functions

.. Linear functionσ(x) = c x

.. Sigmoid functionσ(x) = (1 + e−x)−1

.. Rectified Linear Unit (ReLU)functionσ(x) = max(0, x)

.. Sinusoid functionσ(x) = sin(x)

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.

..13 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksTraining

Given a training set D = (xi, yi)|i = 1, . . . , n, we minimize the lossfunction

L(W,b) =1

2

n∑j=1

∥y(xj ;W,b)− yj∥2,

where y(xj ;W,b) is the output of the overall network function.The commonly used approach to minimize the loss function is thestochastic gradient descent (SGD) approach.

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, Backpropagationapplied to handwritten zip code recognition, Neural Computation, 1(4), 541-551, 1989.

..14 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksTraining

Given a training set D = (xi, yi)|i = 1, . . . , n, we minimize the lossfunction

L(W,b) =1

2

n∑j=1

∥y(xj ;W,b)− yj∥2,

where y(xj ;W,b) is the output of the overall network function.The commonly used approach to minimize the loss function is thestochastic gradient descent (SGD) approach.

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, Backpropagationapplied to handwritten zip code recognition, Neural Computation, 1(4), 541-551, 1989.

..14 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksTraining

Given a training set D = (xi, yi)|i = 1, . . . , n, we minimize the lossfunction

L(W,b) =1

2

n∑j=1

∥y(xj ;W,b)− yj∥2,

where y(xj ;W,b) is the output of the overall network function.The commonly used approach to minimize the loss function is thestochastic gradient descent (SGD) approach.

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, Backpropagationapplied to handwritten zip code recognition, Neural Computation, 1(4), 541-551, 1989.

..14 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksTraining

Given a training set D = (xi, yi)|i = 1, . . . , n, we minimize the lossfunction

L(W,b) =1

2

n∑j=1

∥y(xj ;W,b)− yj∥2,

where y(xj ;W,b) is the output of the overall network function.The commonly used approach to minimize the loss function is thestochastic gradient descent (SGD) approach.

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, Backpropagationapplied to handwritten zip code recognition, Neural Computation, 1(4), 541-551, 1989.

..14 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Artifitial Neural NetworksTraining

Given a training set D = (xi, yi)|i = 1, . . . , n, we minimize the lossfunction

L(W,b) =1

2

n∑j=1

∥y(xj ;W,b)− yj∥2,

where y(xj ;W,b) is the output of the overall network function.The commonly used approach to minimize the loss function is thestochastic gradient descent (SGD) approach.

C. M. Bishop, Pattern Recognition and Machine Learning, springer, 2006.Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, Backpropagationapplied to handwritten zip code recognition, Neural Computation, 1(4), 541-551, 1989.

..14 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network1. Inference Problems 2. Identification Problems

...4 Future Works

..15 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsProblem Setup and Solution Methodology

Considering BHCP

ut(x, t)− κ(t)uxx(x, t) = 0, (x, t) ∈ Ω× [0, T ],u(x, T ) = g(x), x ∈ Ω,

with Dirichlet or Neumann boundary condition, we proceed by approximating thesolution u with DNN and define a model r to be given by

r(x, t) := ut(x, t)− κ(t) uxx(x, t)

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..16 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsProblem Setup and Solution Methodology

Considering BHCP

ut(x, t)− κ(t)uxx(x, t) = 0, (x, t) ∈ Ω× [0, T ],u(x, T ) = g(x), x ∈ Ω,

with Dirichlet or Neumann boundary condition, we proceed by approximating thesolution u with DNN and define a model r to be given by

r(x, t) := ut(x, t)− κ(t) uxx(x, t)

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..16 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = et−4 sin(x) + e4(t−4) sin(2x),we are interested in learning u(x, 0).

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..17 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = et−4 sin(x) + e4(t−4) sin(2x),we are interested in learning u(x, 0).

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..17 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = et−4 sin(x) + e4(t−4) sin(2x),we are interested in learning u(x, 0).

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..17 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = et−4 sin(x) + e4(t−4) sin(2x),we are interested in learning u(x, 0).

Figure : Representation of u(x, t) when (a) T = 1 and (b) T = 4.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..17 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nu

Nu∑i=1

|ui − u(xi, ti)|2 +1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..18 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nu

Nu∑i=1

|ui − u(xi, ti)|2 +1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..18 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nu

Nu∑i=1

|ui − u(xi, ti)|2 +1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..18 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nu

Nu∑i=1

|ui − u(xi, ti)|2 +1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..18 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Representing the solution u by 8-layer DNN with 20 neurons per hedden layer,Nu = 400 and Nr = 6000 when T = 1; and 4-layer DNN with 100 neurons perhedden layer, Nu = 400 and Nr = 20000 when T = 4, we can have the followingresults.

Figure : Representation of u(x, t) (a, c) exact solution, (b, d) predicted solution.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..19 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Representing the solution u by 8-layer DNN with 20 neurons per hedden layer,Nu = 400 and Nr = 6000 when T = 1; and 4-layer DNN with 100 neurons perhedden layer, Nu = 400 and Nr = 20000 when T = 4, we can have the followingresults.

Figure : Representation of u(x, t) (a, c) exact solution, (b, d) predicted solution.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..19 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Table : relative L2-errors in the initial time and whole domain

T = 1 T = 4

noise value E0 E E0 E

0.00 3.22e− 2 1.77e− 2 2.46e− 2 7.52e− 40.01 3.79e− 2 1.52e− 2 6.37e− 2 3.39e− 30.07 5.97e− 2 2.09e− 2 5.11e− 1 2.21e− 20.10 3.46e− 2 9.68e− 3 8.58e− 1 3.22e− 2

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..20 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Dirichlet boundary conditions

Figure : Representation of u(x, 0) with noise (a) 0.00, (b) 0.01, (c) 0.07, (d) 0.10.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..21 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 1: Comparing with RBF method (multiquadric basis function ϕ(r) =

√r2 + c2)

Figure : Representation of u(x, 0) with different shape parameters.

M. Li, T. Jiang, Y.C. Hon, A meshless method based on RBFs method for nonhomogeneous BHCP, EngineeringAnalysis with Boundary Elements, 34:785-792, 2010.

..22 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Considering the heat equation in one space dimension as

ut(x, t)− uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],ux(0, t) = 0 = ux(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = exp(−t) cos(x),we are interested in learning u(x, 0).

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..23 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nb

Nb∑i=1

|(ux)i(x)−ux(x, ti)|2+1

NT

NT∑i=1

|ui−u(xi, T )|2+1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..24 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nb

Nb∑i=1

|(ux)i(x)−ux(x, ti)|2+1

NT

NT∑i=1

|ui−u(xi, T )|2+1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..24 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nb

Nb∑i=1

|(ux)i(x)−ux(x, ti)|2+1

NT

NT∑i=1

|ui−u(xi, T )|2+1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..24 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nb

Nb∑i=1

|(ux)i(x)−ux(x, ti)|2+1

NT

NT∑i=1

|ui−u(xi, T )|2+1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..24 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Representing the solution u by 4-layer DNN with 100 neurons per hedden layer,Nb = 50, NT = 50 and Nr = 100 when T = 1, we can have the following results.

Figure : Representation of u(x, t) (a) exact solution, (b) predicted solution.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..25 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Table : relative L2-errors in the initial time and whole domain

noise value E0 E

0.00 2.26e− 3 1.32e− 30.01 3.68e− 3 3.21e− 30.07 2.21e− 2 2.16e− 20.10 3.04e− 2 3.16e− 2

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..26 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Neumann boundary conditions

Figure : Representation of u(x, 0) with noise (a) 0.00, (b) 0.01, (c) 0.07, (d) 0.10.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..27 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 2: Comparing with RBF methods (multiquadric basis function ϕ(r) =

√r2 + c2)

Figure : Representation of u(x, 0) with different shape parameters.

M. Li, T. Jiang, Y.C. Hon, A meshless method based on RBFs method for nonhomogeneous BHCP, EngineeringAnalysis with Boundary Elements, 34:785-792, 2010.

..28 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 3: Linear time-dependent thermal diffusivity factor

Considering the heat equation with linear time-dependent thermal diffusivityfactor as

ut(x, t) + (2 t+ 1)uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = exp(t(t+ 1)) sin(x)/ exp(2),we are interested in learning u(x, 0).

Figure : Representation of u(x, t) when the final time is T = 1.F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..29 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 3: Linear time-dependent thermal diffusivity factor

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nb

Nb∑i=1

|ui(x)−u(x, ti)|2+1

NT

NT∑i=1

|ui−u(xi, T )|2+1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..30 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 3: Linear time-dependent thermal diffusivity factor

Representing the solution u by 4-layer DNN with 100 neurons per hedden layer,Nb = 50, NT = 100 and Nr = 200 when T = 1, we can have the followingresults.

Figure : Representation of u(x, t) (a) exact solution, (b) predicted solution.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..31 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 3: Linear time-dependent thermal diffusivity factor

Table : relative L2-errors in the initial time and whole domain

noise value E0 E

0.00 1.44e− 3 2.59e− 40.01 3.73e− 3 1.04e− 30.07 8.64e− 3 6.39e− 30.10 1.20e− 2 8.58e− 3

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..32 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 3: Linear time-dependent thermal diffusivity factor

Figure : Representation of u(x, 0) with noise (a) 0.00, (b) 0.01, (c) 0.07, (d) 0.10.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..33 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 3: Comparing with RBF method (multiquadric basis function ϕ(r) =

√r2 + c2)

Figure : Representation of u(x, 0) with different shape parameters.

M. Li, T. Jiang, Y.C. Hon, A meshless method based on RBFs method for nonhomogeneous BHCP, EngineeringAnalysis with Boundary Elements, 34:785-792, 2010.

..34 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 4: Non-linear time-dependent thermal diffusivity factor

Considering the heat equation in one space dimension as

ut(x, t)− (100 + exp(t2))−1uxx(x, t) = 0, (x, t) ∈ [−10, 10]× [0, T ],u(−10, t) = 0 = u(10, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), −10 ≤ x ≤ 10,

with the exact solution

g(x, t) := u(x, t) = exp(−|x|) (cosh(µt(0)) + sinh(µt(0)))

where µt(0) =∫ t

0(100 + exp(t2))−1ds, we are interested in learning u(x, 0).

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..35 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 4: Non-linear time-dependent thermal diffusivity factor

Parameters of the neural networks u can be learned by minimizing the meansquared error loss

MSE =1

Nb

Nb∑i=1

|ui(x)−u(x, ti)|2+1

NT

NT∑i=1

|ui−u(xi, T )|2+1

Nr

Nr∑i=1

|r(xi, ti)|2.

Figure : Representation of u(x, t) when T = 1.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..36 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 4: Non-linear time-dependent thermal diffusivity factor

Representing the solution u by 4-layer DNN with 100 neurons per hedden layer,Nb = 50, NT = 100 and Nr = 20000 when T = 1, we can have the followingresults.

Figure : Representation of u(x, t) (a) exact solution, (b) predicted solution.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..37 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 4: Non-linear time-dependent thermal diffusivity factor

Table : relative L2-errors in the initial time and whole domain

noise value E0 E

0.00 2.47e− 01 3.05e− 020.01 3.07e− 1 5.17e− 30.07 4.96e− 1 7.86e− 20.10 6.54e− 1 1.13e− 1

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..38 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 4: Non-linear time-dependent thermal diffusivity factor

Figure : Representation of u(x, 0) with noise (a) 0.00, (b) 0.01, (c) 0.07, (d) 0.10.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven solutions of BHEs, under preparation.

..39 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Inference ProblemsExample 4: Comparing with RBF methods (multiquadric basis function ϕ(r) =

√r2 + c2)

Figure : Representation of u(x, 0) with different shape parameters.

M. Li, T. Jiang, Y.C. Hon, A meshless method based on RBFs method for nonhomogeneous BHCP, EngineeringAnalysis with Boundary Elements, 34:785-792, 2010.

..40 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsProblem Setup and Solution Methodology

Considering BHCP

ut(x, t)− λuxx(x, t) = 0, (x, t) ∈ Ω× [0, T ],u(x, T ) = g(x), x ∈ Ω,

with Dirichlet or Neumann boundary condition, we proceed by approximating thesolution u with DNN and define a model r to be given by

r(x, t) := ut(x, t)− λ uxx(x, t)

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..41 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsProblem Setup and Solution Methodology

Considering BHCP

ut(x, t)− λuxx(x, t) = 0, (x, t) ∈ Ω× [0, T ],u(x, T ) = g(x), x ∈ Ω,

with Dirichlet or Neumann boundary condition, we proceed by approximating thesolution u with DNN and define a model r to be given by

r(x, t) := ut(x, t)− λ uxx(x, t)

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..41 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + λ uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = exp(−t) cos(x),we are interested in learning u(x, 0) and λ. Parameters of the neural networks uand λ can be learned by minimizing the mean squared error loss

MSE = 1Nb

∑Nb

i=1 |ui(x)− u(x, ti)|2 + 1NT

∑NT

i=1 |ui − u(xi, T )|2

+ 1Nr

∑Nr

i=1 |r(xir, t

ir)|2 + 1

Nr

∑Nr

i=1 |uir − u(xi

r, tir)|2.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..42 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + λ uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = exp(−t) cos(x),we are interested in learning u(x, 0) and λ. Parameters of the neural networks uand λ can be learned by minimizing the mean squared error loss

MSE = 1Nb

∑Nb

i=1 |ui(x)− u(x, ti)|2 + 1NT

∑NT

i=1 |ui − u(xi, T )|2

+ 1Nr

∑Nr

i=1 |r(xir, t

ir)|2 + 1

Nr

∑Nr

i=1 |uir − u(xi

r, tir)|2.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..42 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + λ uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = exp(−t) cos(x),we are interested in learning u(x, 0) and λ. Parameters of the neural networks uand λ can be learned by minimizing the mean squared error loss

MSE = 1Nb

∑Nb

i=1 |ui(x)− u(x, ti)|2 + 1NT

∑NT

i=1 |ui − u(xi, T )|2

+ 1Nr

∑Nr

i=1 |r(xir, t

ir)|2 + 1

Nr

∑Nr

i=1 |uir − u(xi

r, tir)|2.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..42 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsExample 1: Dirichlet boundary conditions

Considering the heat equation in one space dimension as

ut(x, t) + λ uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],u(0, t) = 0 = u(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = exp(−t) cos(x),we are interested in learning u(x, 0) and λ. Parameters of the neural networks uand λ can be learned by minimizing the mean squared error loss

MSE = 1Nb

∑Nb

i=1 |ui(x)− u(x, ti)|2 + 1NT

∑NT

i=1 |ui − u(xi, T )|2

+ 1Nr

∑Nr

i=1 |r(xir, t

ir)|2 + 1

Nr

∑Nr

i=1 |uir − u(xi

r, tir)|2.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..42 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsExample 1: Dirichlet boundary conditions

Table : relative L2-errors in the initial time and whole domain and unknowncoefficent

T = 1 T = 4

N.V E0 E λ E0 E λ

0.00 3.51e− 2 1.38e− 2 0.958 2.60e− 2 1.02e− 3 1.0010.01 4.34e− 2 1.62e− 2 0.956 2.41e− 2 3.13e− 3 0.9970.07 4.77e− 2 1.98e− 2 0.938 1.04e− 2 7.76e− 3 1.0010.10 6.03e− 2 2.62e− 2 0.913 2.64e− 1 9.51e− 3 1.002

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..43 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsExample 2: Neumann boundary conditions

Considering the heat equation in one space dimension as

ut(x, t)− λ uxx(x, t) = 0, (x, t) ∈ [0, π]× [0, T ],ux(0, t) = 0 = ux(π, t), 0 ≤ t ≤ T,u(x, T ) = g(x, T ), 0 ≤ x ≤ π,

with the exact solution g(x, t) := u(x, t) = et−4 sin(x) + e4(t−4) sin(2x),we are interested in learning u(x, 0) and λ. Parameters of the neural networks ucan be learned by minimizing the mean squared error loss

MSE = 1Nb

∑Nb

i=1 |(ux)i(x)− ux(x, ti)|2 + 1NT

∑NT

i=1 |ui − u(xi, T )|2

+ 1Nr

∑Nr

i=1 |r(xir, t

ir)|2 + 1

Nr

∑Nr

i=1 |uir − u(xi

r, tir)|2.

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..44 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

BHE & DNN: Identification ProblemsExample 1: Neumann boundary conditions

Table : relative L2-errors in the initial time and whole domain and unknowncoefficent

noise value E0 E λ

0.00 1.72e− 3 9.42e− 4 0.9990.01 6.28e− 3 3.87e− 3 0.9990.07 3.92e− 2 2.32e− 2 0.9960.10 5.77e− 2 3.36e− 2 0.996

F. Mostajeran, S. M. Hosseini, R. Mokhtari, Using deep learning for data-driven discovery of BHEs, under preparation.

..45 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Outline

...1 Introduction

...2 Artifitial Neural Networks

...3 Backward Heat Equation and Deep Neural Network

...4 Future Works

..46 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Future Works

Does DNN have the following advantages?.. extendable to solve high-dimensional problems,.. applicable to solve more general class of problems defined on irregular

domains,.. applicable to learn κ(t) or more complicated κ(x, t).

..47 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Future Works

Does DNN have the following advantages?.. extendable to solve high-dimensional problems,.. applicable to solve more general class of problems defined on irregular

domains,.. applicable to learn κ(t) or more complicated κ(x, t).

..47 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Future Works

Does DNN have the following advantages?.. extendable to solve high-dimensional problems,.. applicable to solve more general class of problems defined on irregular

domains,.. applicable to learn κ(t) or more complicated κ(x, t).

..47 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Future Works

Does DNN have the following advantages?.. extendable to solve high-dimensional problems,.. applicable to solve more general class of problems defined on irregular

domains,.. applicable to learn κ(t) or more complicated κ(x, t).

..47 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.

Thanks for Your Attention!Comments are welcome.f_mostajeran@modares.ac.ir

..47 / 47

.Using DNN for Solving BHEs, F. Mostajeran, Summer School in Graz

.