Advanced Space Weather Modeling: Year 1

Gabor Toth (PI), Ward Manchester (Co-PI), Bart van der Holst (CoI),

Yuxi Chen (graduated student/post doc), Tamas Gombosi(University of Michigan)

Paul Cassak (West Virginia University)Vania Jordanova (LANL)

Stefano Markidis, Ivy Bo Peng (KTH, Sweden)

NSF INSPIRE, PRAC Los Alamos SHIELDS

2

Motivation and Outline

Breakthrough advance in the understanding of space weatherSpace weather involves dynamical processes of the Sun-Earth system that affect human life and technology (radiation, GPS, blackouts…)Use innovative algorithms and models in combination with high-end computing facilities (BW)

Coronal Mass EjectionCauses the most destructive space weather eventsInitiated from active regionsInvolves magnetic flux emergence from the convection zone into the corona, but the details of the process are not understoodSpherical Wedge Active Region Model (SWARM)

Geomagnetic StormsResults from the interaction of the varying solar wind and interplanetary magnetic field (often caused by CME) with the magnetosphereMagnetic reconnection happening at kinetic scales plays a crucial roleMagnetohydrodynamics with Embedded Particle-in-Cell (MHD-EPIC)

Space Weather Modeling FrameworkCouples the models together to simulate the whole system

3

SWMF Control & Infrastructure

EruptionGenerator

Solar Corona

InnerHeliosphere

GlobalMagnetosphere

Polar Wind

InnerMagnetosphere

IonosphericElectrodynamics

Thermosphere& Ionosphere

EnergeticParticles

Radiation

Belts

3D OuterHeliosphere

SWMF is freely available at http://csem.engin.umich.edu and via CCMC

Couplers

Flare/CMEObservations

UpstreamMonitors

RadarsMagnetometers

In-situ

F10.7 FluxGravity Waves

Space Weather Modeling Framework

Magneto-grams,rotation

tomography

Particle in Cell

ParticleTracker

ConvectionZone

RCM CRCM

RAM-SCB HEIDI

BATSRUS

PWOM

GITM

RBE

RIM

IPIC3D, ALTOR

AMPS

BATSRUS

BATSRUS

KotaMFLAMPA

BATSRUS

BATSRUSFSAM

PhysicsClassical, semi-relativistic and Hall MHD Multi-species, multi-fluid, 5-momentanisotropic pressure for ion fluidsMulti-material, non-ideal equation of stateHeat conduction, viscosity, resistivityAlfven wave turbulence and heatingRadiation hydrodynamics multigroup diffusion

NumericsParallel Block-Adaptive Tree Library (BATL)Cartesian and generalized coordinatesSplitting the magnetic field into B0 + B1Divergence B control: 8-wave, CT, projection, parabolic/hyperbolicNumerical fluxes: Godunov, Rusanov, AW, HLLE, HLLD, Roe, DWExplicit, local time stepping, limited time step, sub-cyclingPoint-, semi-, part and fully implicit time steppingUp to 4th order accurate in time and 5th order in space

ApplicationsHeliosphere, sun, planets, moons, comets, HEDP experiments

150,000+ lines of Fortran 90+ code with MPI parallelization 4

BATS-R-US

Active Regions and Coronal Mass Ejections

5

Spherical Wedge Active Region Model (SWARM)on Blue Waters

6

Spherical wedge gridGravity: 1/r2

Tabular EOS ionizationRadiative cooling35 Mm deep (0.95 Rs)5 levels of refinement10x10x10 cell blocks160 million grid cells3 million time steps 288 hours on 16,384 cores:~5 million CPU hours

Convection and

Granulation

SWARM on BW

Observed solar granulation

Active Region Scale Flux Emergence Simulation

Add toroidal magnetic flux rope Twist factor =11023 Mx flux20 Mm deep (0.9714 Rs)

Magnetic Flux Emergence near the Photosphere

R = 1 RsMagnetic field distribution dominated by convectionFlux concentrated in downdraftsMagnetic field evolves parallel to polarity inversion lineShear flows driven by the Lorentz force

Next Step: SWARM + AWSoM Simulations

Realistic size active region model (SWARM)Global solar corona model (AWSoM)2-way coupling at every time stepAllow erupting flux to expand into coronaSelf-consistent CME initiation

12

Solar Wind-Magnetosphere Interaction

wind

solar wind

Solar wind is a stream of ions and electrons originating from the solar upper atmosphere. The solar wind is accompanied with the interplanetary magnetic field (IMF). The solar wind compresses Earth’s magnetic field on the dayside, and stretches the field lines on the tail side: forms the magnetosphere

13

Magnetospheric Dynamics

Magnetic reconnection happens at both the dayside magnetopause and magnetotailMagnetopause reconnection transfers mass and energy from the solar wind to the magnetosphereMagnetotail reconnection releases the energy stored in the magnetosphere

From Eastwood et al. (2015)

14

MHD with Embedded Particle-in-Cell (MHD-EPIC) in a Nutshell

The goal is to combine the efficiency of the global (extended) MHD code with the physics capabilities of the local PIC code!

MHD code - BATS-R-USSolves the ideal, resistive, Hall, two-fluid, multi-ion, anisotropic ion pressure MHD equations on 1, 2 or 3D block-adaptive Cartesian and non-Cartesian gridsComplicated initial and boundary conditions, variety of source termsF90+

PIC code - IPIC3DSolves full set of Maxwell's equations implicitly on 3D Cartesian gridEquations of motion for electrons and ion speciesC++

Framework – SWMFExecutes and couples multiple models on massively parallel machinesEfficient parallel coupler developed for MHD-EPIC algorithmMultiple PIC domains

15

Two-way Coupled MHD-EPIC Algorithm

BATS-R-USt = 0

iPIC3Dt = 0

PIC region, initial and boundary conditions

t = Δtcouplet = Δtcouple

PIC solution in PIC region

MHD boundary conditions

Challenge of Global vs Kinetic Scales

Hypothesis: Overall reconnection dynamics is correctly captured as long as the global, ion and electron scales are well separated and the model captures each scale and has the appropriate physics.What are the consequences? What are the limitations?

16

200

400

600

800

Ion inertial length (km)

-50 -40 -30 -20 -10 0 10 20X

-30

-20

-10

0

10

20

30

Z

X [Re]

Z[R

e]

Explicit PIC codes need to resolve the Debye length. Standard trick: decrease the speed of light.Reconnection happens at the electron scales proportional to the electron skin depth de. Trick: increase the electron mass to make de larger.The ion scales are proportional to the ion inertial length di that is about 60 km ~ 1/100RE in the magnetosheath: Too small to resolve! Even with MHD-EPIC even on BW!Trick: reduce q/M by a scaling factor f to increase di and de.

17

Dayside

2D simulations:ρSW= 5 amu/cm3

uSW= 400 km/spSW= 0.031 nPape,SW = 0.024 nPaBSW=-0.1,-0.5,-3 nTdi(f=8) ~ 1/12 Re

di(f=32) ~ 1/3 Re

Mi/Me=100, de=di/10Δx = 1/64 Re

Global reconnection

dynamics is not sensitive to the scaling factor f!

MHD-EPIC f = 8

2-fluid MHD f = 32

2-fluid MHD f = 8

MHD-EPIC f = 32

18

Smallscale

Electron velocity components near reconnection sites in two MHD-EPIC simulations

Local dynamics length scale

scales with f !

MHD-EPIC f = 8, Δx = 1/128 Re, ¾ Re x ¾ Re zoom-in

MHD-EPIC f = 32, Δx = 1/32 Re, 3 Re x 3 Re zoom-in

19

3D MHD-EPIC Simulation Setup

Physical parametersUse f =16 so di ~ 1/6 RE

Typical solar wind conditions: ρ= 5 amu/cm3, UX = -400km/s, B = [0,0,-5] nT

Hall MHD with separate electron pressure equation

MHD domain: -224 < x < 32, -128 < y, z < 128RE

At the magnetopause Δx=1/16RE (~400km)MHD uses ~20% CPU time

PICPIC domain: 8 < x < 12, -6 < y < 6, -6 < z < 6RE

Δx = 1/32RE: 5 cells per di (for f = 16)216 particles per cell per species: 8B total Consuming ~80% simulation time

1 hour long simulation

180 hours on 6,400 cores: ~1.2M core hours

p [nPa]

20

Evolution of FTEs (1)t=100s t=150s

t=240s t=320s

t=540s t=660s

FTE-A

FTE-A

FTE-B

FTE-A

FTE-B

FTE-C

FTE-B

FTEs colored by uizIMF is purely southwardFTEs grow in the dawn-dusk directionuiz varies along the FTE, and the FTE becomes tiltedTwo FTEs can merge into one

21

Crater FTE

10

20

30

40

50

60

70B t

9.5 10.0 10.5 11.0 11.5X

-2

-1

0

1

2

3

4

Z

9.5 10.0 10.5 11.0 11.5X

-2

-1

0

1

2

3

4

Z

9.5 10.0 10.5 11.0 11.5X

-2

-1

0

1

2

3

4

Z

9.5 10.0 10.5 11.0 11.5X

-2

-1

0

1

2

3

4

Z

-1 0 1 2 3 4z

-30-10

1030

B x

-1 0 1 2 3 4z

-20-1001020

B z

-1 0 1 2 3 4z

-30-1501530

B y

-1 0 1 2 3 4z

010

2030

B t

THEMIS dataSimulation

B field along the red dashed line

FTE from simulationBx jumps from the negative peak (z=0) to the positive peak (z=1RE)Bounded by the depressed magnetic field `trenches' at z=-0.2RE and z =2RE

Bt reaches local minimum at the center

22

Tripolar Guide Field

-1 0 1 2 3z

-30-20

-10

0

1020

BM

-30

-20

-10

0

10

20

30BM

8 9 10 11 12-6

-4

-2

0

2

4

6

Z

x

Polar observation

Simulation

23

Crescent Distribution and Larmor Electric Field

Ion

-600 -400 -200 0 200 400 600Vy (km/s)

-600

-400

-200

0

200

400

600

Vx (

km

/s)

0.0

0.2

0.4

0.6

0.8

1.0

Electron

-4000 -2000 0 2000 4000Vy (km/s)

-4000

-2000

0

2000

4000

Vx (

km

/s)

0.0

0.2

0.4

0.6

0.8

1.0

-5

0

5

10

15Ex

9.0 9.5 10.0 10.5 11.0

X

-4

-3

-2

-1

0

Z

Ex

9.0 9.5 10.0 10.5 11.0

X

-5

0

5

10

15

(a) (b)

(c)

(d)

MMS observation

Burch et al. Science, 2016

Ex

9.0 9.5 10.0 10.5 11.0X

-5051015

24

Lower Hybrid Drift Instability (LHDI)

-10

-5

0

5

10

9 10 11X

-4

-2

0

2

4

Y

9 10 11X

-4

-2

0

2

4

Y

-5

0

5

10

9.5 10.0 10.5 11.0X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y

9.5 10.0 10.5 11.0X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y

-20

0

20

40

9.5 10.0 10.5 11.0X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y

9.5 10.0 10.5 11.0X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y

-1200

-1000

-800

-600

-400

-200

0

9.5 10.0 10.5 11.0X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y

9.5 10.0 10.5 11.0X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y

5

10

15

9.5 10.0 10.5 11.0X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y9.5 10.0 10.5 11.0

X

-1.0

-0.5

0.0

0.5

1.0

1.5

Y

(a) EM (b) EM (c) Bz

(d) ρi (e) uey

(f) uiz

LHDI arises near the interface of magnetosheath and magnetosphere, where there is a sharp density gradient Simulation agrees with MMS observation

EM ~ 8 mV/mkre ~ 0.4, λ ~ 16 re

25

Summary

Spherical Wedge Active Region Model (SWARM)Largest simulation of an active region with proper geometryConvection zone physics produced wellFlux emergence simulation is in progress

Kinetic scalingChanging the ion charge per mass can make coupled global-kinetic simulations affordable. Detailed parameter study performed on BW.Toth et al. 2017 submitted to JGR.

MHD-EPIC simulation of Earth’s magnetosphereDay side reconnection is modeledFirst two-way coupled MHD-kinetic simulation of Earth’s magnetosphereFlux Transfer Events formed by reconnection are simulated: good agreement with in-situ measurementsKinetic features: crescent electron phase space distribution, Larmor electric field, lower hybrid drift instability (LHDI). Agrees with MMS observations.Chen et al. 2017 submitted to JGR

26

Why Blue Waters & Future WorkWhy Blue Waters?

Simple and easy accessadding users takes a daylogin procedure is simplefile transfer is made easySupport staff is knowledgable and helpful

Order of magnitude more allocation than on other systemsReasonably short turn-around times for large runsGood software environment, stable hardware

Future workSWARM + AWSoM simulations:

Complete flux emergence simulation Continue simulation coupled to global corona model

MHD-EPIC simulations of Earth’s magnetosphere:Do actual events and compare with MMS observationsCover both magnetopause and magnetotail with PIC boxes to study magnetic storms and substorms