Yuri Mishin, George Mason university

1. 1
G. P. Purja Pun1, R. Batra2, R. Ramprasad3 and Y. Mishin1
Dept of Physics and Astronomy, George Mason University, Fairfax, VA 22030
Dept of Physics and Astronomy, George Mason University, Fairfax, VA
Dept of Materials Science and Engineering, University of Connecticut, Storrs, CT
School of Materials Science and Engineering, Georgia Institute of Technology,
Atlanta, GA
J. Hickman, K. Choudhary and F. Tavazza
MSED, MML, National Institute of Standards and Technology, Gaithersburg, MD
V. Yamakov and E. H. Glaessgen
NASA Langley Research Center, Hampton, VA
Classical force fields as physics-based neural
networks
Sponsored by the Office of Naval Research
Workshop Artificial Intelligence for Materials Science, NIST, August, 2018

2. 2
Outline
๏ Atomistic simulations with classical interatomic potentials
• Traditional potentials: strengths and shortcomings
๏ Machine-learning potentials
• NN potentials: strengths and shortcomings
๏ Physically-informed neural network (PINN) potentials
• General idea
• PINN Al potential
๏ Future work

3. 3
Atomistic simulations of materials
✦ Explicit modeling of atoms/molecules
๏ Molecular dynamics
๏ Monte Carlo
✦ Interatomic potentials (a.k.a. classical force fields)
๏ Parameterize configuration space of the system. Express total
energy and classical forces as a function of atomic coordinates
๏ Contain adjustable parameters that need calibration
๏ Very fast and scale as ~N (DFT ~N3) access to large systems
(~106 atoms, ~10 ns)
๏ Modern potentials: angular-dependent, environmentally-dependent,
with reactive functionals, many fitting parameters. Often represent
materials properties on a quantitative level

4. 4
Traditional interatomic potentials
✦ Specific to material: metals (EAM, MEAM, ADP), covalent (Tersoff, SW),
molecular systems and reactions (ReaxFF).
✦ General-purpose usage
๏ Thermodynamic properties (phase diagrams, interface free energies)
๏ Mechanical properties (plastic deformation, fracture)
๏ Diffusion kinetics
10
-12
10-11
10
-10
1.0 1.1 1.2 1.3 1.4
1000 950 900 850 800 750
Dd[m2
s-1
]
Tm/T
T [K]
Intrinsic
Vacancy
Interstitial
Cu-Ag phase diagram Dislocation pipe diffusion in Al
Deformation of nanocrystalline Al

5. 5
Traditional interatomic potentials
p = (p1, p2, . . . , pm)
i
jrij
✦ Partition the total energy:
✦ Local mapping:
✦ Function:
E =
∑
i
Ei
(r1, . . . , rn)i ⟼ Ei
Ei = Ei(r1, . . . , rn, p)
with adjustable parameters (usually, 10-20)
✦ Fit parameters to a small database of experimental and DFT data
✦ Direct fit to properties (not just energies): E0, a0, cij, Ev,…
✦ Important: the functional form is motivated by physical/chemical intuition
Ei

6. 6
Traditional interatomic potentials: Pros and cons
• Pros:
๏ Very fast. Afford simulations of ~106 atoms for ~102 ns
(or longer with accelerated MD)
๏ Based on physics reasonable transferability
๏ Inaccurate but not crazy
• Cons:
๏ Few parameters, small training dataset inaccurate
๏ Cannot be improved systematically
๏ Specific to given class of materials
๏ Development is painfully difficult and slow. Heavily relies
on human experience. More art than science

7. 7
Machine-learning interatomic potentials
✦ First introduced by chemists in the 1990s
✦ Direct mapping on the potential energy surface (PES):
(r1, . . . , rn)i ⟼ (G1, . . . , Gm)i ⟼ Ei
✦ “Fingerprints” invariant under
rotations and translations of coordinates
✦ Nonlinear regression with ~103 parameters to fit
✦ Training on a large DFT database with 103-104
supercells
✦ High accuracy of fit: ~ 1-5 meV/atom (DFT level)
✦ Purely mathematical interpolation between the
training points. No guidance from physics or
chemistry
(G1, . . . , Gm)i
i
jrij
Ei
(G1, . . . , Gm)i
Regression

8. 8
Mathematical NN potentials
• Example (Behler and Parrinello, PRL 2007): NN potential for Si
• 48 structural parameters (Gi’s)
• NN with 48-40-40-1 architecture, ~2000 parameters
• Training set of ~10,000 DFT energies for ~10 crystals structures of
Si at different T and p
• 5 meV/atom fitting, 5-6 meV/atom testing. 104 faster than DFT
Atom i
Neural
network
Other
atoms
Ei Σ PES
Local structural
parameters
w1 w2
b1
b2
Input
layer
Output
layer
Hidden
layer
p q
Fitting parameters:
Weights wij
biases bi

9. 9
Machine-learning potentials: Pros and cons
✦ Advantages
๏ Extremely accurate (~meV/atom - DFT level)
๏ Much faster than DFT. “Accelerated DFT”?
๏ No physics applicable to any type of bonding, including mixed
metallic-covalent
๏ Can be improved systematically with more DFT data
✦ Drawbacks
๏ Much slower than traditional IPs
๏ No standard software (like LAMMPS)
๏ Requires massive DFT calculations
๏ Can only interpolate between the DFT energies. Cannot extrapolate
outside the training dataset. Poor transferability

10. 10
Physically-informed NN (PINN) potentials
Taking the best from both worlds
• IPs are locally very accurate. Different structures can be fit
with different sets of parameters
• Idea: Make parameters functions of local environment
Atom i
Neural
network
Potential
parameters
Other
atoms
Interatomic
potential Ei
Σ PES
Neighboring
atoms
Local structural
parameters
๏ Extrapolation uses physical insights
๏ Transferability is improved
Potential parameters
adjusted on the fly

11. 11
PINN potentials: Choice of potential
Atom i
Neural
network
Potential
parameters
Other
atoms
Interatomic
potential Ei
Σ PES
Neighboring
atoms
Local structural
parameters ?
✦ Requirements for the potential model:
๏ Reflect general properties of chemical bonding
๏ General enough to include different classes of materials

12. 12
Analytical bond order potential
Total energy: E =
∑
i
Ei
Atomic energy: Ei =
1
2 ∑
j≠i
[eAi−αirij − SijbijeBi−βirij
] fc(rij) + E(p)
i
Include 4-5 coordination shells!
Bond order:
Coordination number:
bij = (1 + zij)−1/2
zij = a2
i ∑
k≠i,j
Sik(cos θijk + hi)2
fc(rik)
i
k
jrij
Bond screening:
Sij =
∏
k≠i,j
Sijk
Sijk = 1 − fc(rik + rjk − rij)e−λ2
i (rik+rjk−rij)
✦ Physical effects
๏ Short-range repulsion
๏ Bond order (more neighbors - weaker bond)
๏ Bond-angle dependence
๏ Bond screening by neighbors
✦ 8 parameters
✦ Applicable to covalent and
metallic materials

13. 13
Comparison of potentials
• Inaccurate
• Physically meaningful
• Accurately trained
• Unpredictable
extrapolation
• Accurately trained
• Physically meaningful
extrapolation
• Interpolation can be
also improved
E
V
E
V
Training
Traditional
E
V
NN PINN
Validation

14. 14
PINN and NN potentials for Al
✦ DFT database: 3154 supercells (108052 atoms)
๏ EOS for 7 crystal structures under tension/compression
๏ Surfaces (100), (110), (111), (311)
๏ Vacancy
๏ Grain boundary, intrinsic stacking fault
๏ Clusters with different sizes and shapes
๏ Several pressures and temperatures
✦ Network architectures
๏ PINN: 60 x 15 x 15 x 8 (1283 parameters)
๏ NN: 60 x 16 x 16 x 1 (1265 parameters)
✦ RMSE of training and testing: ~4.1 meV/atom

15. 15
PINN errors of training and testing
-4
-3
-2
-1
0
1
2
3
4
-4 -3 -2 -1 0 1 2 3 4
Computed energy (eV/atom)
DFT energy (eV/atom)
0
5
10
15
20
25
30
35
-20 -15 -10 -5 0 5 10 15 20
Percentage
EPINN - EDFT (meV/atom)
0
5
10
15
20
25
30
35
-20 -15 -10 -5 0 5 10 15 20
Percentage
EPNN - EDFT (meV/atom)
Training
Testing

16. 16
Al properties: PINN/NN versus DFT
๏ The properties were not fitted to (only the PES was)
๏ Excellent agreement with DFT
๏ PINN performs better than NN

17. 17
EOS of alternate crystal structures
-4.0
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
4.0
0 20 40 60 80 100 120 140
Energy (eV/atom)
Volume (Å3
/atom)
A15
SH
DC
-4.0
-3.0
-2.0
-1.0
0.0
1.0
2.0
3.0
4.0
0 10 20 30 40 50 60 70 80 90 100
Energy (eV/atom)
Volume (Å3
/atom)
HCP
BCC
SC
-4.0
-3.5
-3.0
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
0 20 40 60 80 100 120 140
Energy (eV/atom)
Volume (Å3/atom)
DFT
PINN
NN
E → -0.35 eV
(physically absurd)
FCC

18. 18
Al thermal expansion
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
0 200 400 600 800 1000
Thermallinearexpansion(%)
Temperature (K)
NN
PINN
Experiment
๏ Thermal expansion was not in the DFT database
๏ PINN prediction is more accurate

19. 19
Tests of transferability
-3.3
-3.2
-3.1
-3.3 -3.2 -3.1
Computedenergy(eV/atom)
DFT energy (eV/atom)
Dislocation
-3.3
-3.2
-3.1
-3.3 -3.2 -3.1
Computedenergy(eV/atom)
DFT energy (eV/atom)
Dislocation
-1.0
-0.9
-0.8
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-1.0 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2
Computedenergy(eV/atom)
DFT energy (eV/atom)
HCP
-1.0
-0.9
-0.8
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-1.0 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2
Computedenergy(eV/atom)
DFT energy (eV/atom)
HCP
Al dislocation
700 K
HCP Al
1000-4000 K
PINN
PINN
NN
NN
… and many similar tests

20. 20
Conclusions
๏ ML potentials emerge as next-generation models
๏ Main limitation – transferability. No physics – no
transferability
๏ The PINN model incorporates guidance from physics/
chemistry and is more transferable
๏ PINN potential for Al
๏ PINN Si and Ge in progress (James Hickman)
๏ The PINN model is scalable (~N) but requires new HPC
simulation software. Collaboration with NASA LaRC
๏ Grand challenge: Extension to binary and multicomponent
systems