Scrapbook (Posts about machine-learning)

ZeoNet How-tos

author — Mon, 11 Mar 2024 17:04:09 GMT

Tutorials

Introduction to ZeoNet (slides SharePoint folder, paper, GitLab repository), a representation learning framework using convolutional neural networks and 3D volumetric representations for predicting adsorption Henry in more than 330k zeolites.
Documentation to PyTorch, the main library we used for deep learning tasks.
Tutorials to Pymatgen, a robust python library for materials analysis.

Running jobs on HPC systems using Shotgun

Setup on group server

Set up shotgun on group server.
Complete Step 1 in the Shotgun workflow. For our ZeoNet project, the local_job_root is $HOME/projects/zeonet. The local_asset_dir is $HOME/projects/zeonet/bin. In the local_job_root, clone the ZeoNet repository from the GitLab repository to the local_asset_dir.
```
mkdir -p $HOME/projects/zeonet && cd $HOME/projects/zeonet
shotgun new-project  # and edit shotgun.yaml
git clone git@gitlab.com:baigroup/zeonet.git bin
rm -r bin/ML-Zeolites # remove sample dataset files
```
Create symbolic links for the dataset from the shared group directory.
```
ln -sf /home/user/NOBAK_projects/ML-Zeolites .
```
Upload the local_asset_dir to the HPC. Different HPCs have their own file systems, and we first need to figure out an appropriate filesystem location to be used as the primary location for running HPC jobs. Take Unity as an example, we first SSH to Unity to create a Scratch Workspace (the command below will create /scratch/workspace/user_umass_edu-ABC) to host the work directory. Edit shotgun.yaml to specify this work directory (e.g., /scratch/workspace/user_umass_edu-ABC/zeonet) as job_root.
```
ssh unity ws_allocate ABC  # For Unity only
shotgun upload-asset -t unity-24C-8G
```

Setup on HPC

We need to verify dataset including CIF structure files, volumetric grids HDF5 files, dataset split TXT files, and the zeolite property CSV file existing on HPC. On HPC systems where directories can be shared with the group, the target files usually already existed (e.g., /work/pi_pengbai_umass_edu/ML-Zeolites exist on unity) and, if not, we need to rsync everything ourselves (make sure to read step 4 of Shotgun basic workflow and please contact administrator then.

SSH to the HPC. Install Conda and create a new Conda environment from the environment-linux64-cuda121.txt file.

ssh unity
cd /scratch/workspace/$USER-ABC/zeonet # For Unity
module load conda/latest  # The conda version may be different for other HPCs
module load cuda/12.1
conda env create -f environment-linux64-cuda121.yml

The following steps must be done once. Since the dataset was already put in the shared group directory (e.g., /work/pi_pengbai_umass_edu/ML-Zeolites/), we only need to create symbolic links.
```
ln -sf /work/pi_pengbai_umass_edu/ML-Zeolites .
```

Submitting jobs

Prepare the job submission script. The best practice is to write a generic submission script and put it in the local asset directory (e.g., $HOME/projects/zeonet/bin), and then create symbolic links in each job directory:
```
ln -sf ../bin/sub-zeonet.script .  # Run in current directory, e.g., $HOME/projects/zeonet/BatchSize16-LR0.001-OptAdam
```
Create a config.yaml file. Refer to the configuration files in configs folder.

Submit the job using Shotgun.

shotgun submit -c 1 -g 1 -n 1 -t 336 -e 20G -r unity-24C-8G -j ../bin/sub-zeonet-unity.script

Running an interactive job on HPC systems

With an interactive job, we request time and resources to work on a compute node directly. In this way, we can interact with applications, execute scripts, or run other commands in real time within an HPC environment, which are convenient for debugging, designing workflows, and doing small tests. First SSH to the HPC and then request an interactive job.
```
ssh unity
srun --nodes=1 --ntasks-per-node=1 --cpus-per-task=1 --gpus=1 time=1:00:00 --job-name ML --partition ials-gpu --pty bash  # For Unity
```

Machine-Learning Forcefields

author — Wed, 06 Mar 2024 20:06:32 GMT

Cambridge Gabor Csanyi

MACE

https://github.com/ACEsuit/mace
https://mace-docs.readthedocs.io/_/downloads/en/latest/pdf/
Batatia, I.; Batzner, S.; Kovács, D. P.; Musaelian, A.; Simm, G. N. C.; Drautz, R.; Ortner, C.; Kozinsky, B.; Csányi, G. The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentials,' Nat. Mach. Intell. 7 56-67 (2025).
Batatia, I.; Kovacs, D. P.; Simm, G.; Ortner, C.; Csanyi, G. MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields,' Adv. Neural Inf. Process. Syst. 35 11423–11436 (2022).

QUIP

https://github.com/libAtoms/QUIP
Csányi, G.; Winfield, S.; Kermode, J. R.; De Vita, A.; Comisso, A.; Bernstein, N.; Payne, M. C. Expressive Programming for Computational Physics in Fortran 95+,' IoP Comput. Phys. Newsletter Spring 2007 (2007),
Bartók, A. P., Payne, M. C., Kondor, R., & Csányi, G. Gaussian approximation potentials: The accuracy of quantum mechanics, without the electrons,' Phys. Rev. Lett. 104 136403 (2010).

Miscellaneous

Deringer, V. L., & Csányi, G. Gaussian Approximation Potentials: The Accuracy of Quantum Mechanics, without the Electrons,' Phys. Rev. Lett. 104 136403 (2010).
Bartók, A. P., Kondor, R., & Csányi, G. On representing chemical environments,' Phys. Rev. B 87 219902 (2013).

Harvard Boris Kozinsky

FLARE

https://github.com/mir-group/flare
Vandermause, J., Torrisi, S.B., Batzner, S. et al. On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events. npj Comput Mater 6, 20 (2020).
Vandermause, J., Xie, Y., Lim, J.S. et al. Active learning of reactive Bayesian force fields applied to heterogeneous catalysis dynamics of H/Pt. Nat Commun 13, 5183 (2022).

NequIP

https://github.com/mir-group/nequip
Batzner, S., Musaelian, A., Sun, L. et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat Commun 13, 2453 (2022).

Allegro

https://github.com/mir-group/allegro
Musaelian, A., Batzner, S., Johansson, A. et al. Learning local equivariant representations for large-scale atomistic dynamics. Nat Commun 14, 579 (2023).

M3GNet

https://github.com/materialsvirtuallab/matgl
https://github.com/materialsvirtuallab/maml
Chen, C., Ong, S.P. A universal graph deep learning interatomic potential for the periodic table. Nat Comput Sci 2, 718–728 (2022).

DeePMD-kit

https://github.com/deepmodeling/deepmd-kit
Wang, H.; Zhang, L.; Han, J.; E, W. DeePMD-kit: A Deep Learning Package for Many-Body Potential Energy Representation and Molecular Dynamics, Comput. Phys. Commun. 228 178-184 (2018).
Zeng, J.; Zhang, D.; Lu, D.; Mo, P.; Li, Z.; Chen, Y.; Rynik, M.; Huang, L.; Li, Z.; Shi, S.; Wang, Y.; Ye, H.; Tuo, P.; Yang, J.; Ding, Y.; Li, Y.; Tisi, D.; Zeng, Q.; Bao, H.; Xia, Y., et al. DeePMD-kit v2: A Software Package for Deep Potential Models, J. Chem. Phys. 159 054801 (2023).
Zeng, J.; Zhang, D.; Peng, A.; Zhang, X.; He, S.; Wang, Y.; Liu, X.; Bi, H.; Li, Y.; Cai, C.; Zhang, C.; Du, Y.; Zhu, J.-X.; Mo, P.; Huang, Z.; Zeng, Q.; Shi, S.; Qin, X.; Yu, Z.; Luo, C., et al. DeePMD-kit v3: A Multiple-Backend Framework for Machine Learning Potentials. J. Chem. Theory Comput. 21, 4375–4385 (2025).
https://github.com/deepmodeling/dpgen
https://github.com/deepmodeling/dpgen2
Zhang, Y.; Wang, H.; Chen, W.; Zeng, J.; Zhang, L.; Wang, H.; E, W. DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models,' Comput. Phys. Commun. 253, 107206 (2020).

Miscellaneous

Behler-Parinello

Physical review letters, 98(14):146401, 2007
Physical Chemistry Chemical Physics, 13(40):17930–17955, 2011

ANI-1

Chemical science, 8(4):3192–3203, 2017

SNAP

Journal of Computational Physics, 285:316–330, 2015
Journal of chemical physics, 148(24):241721, 2018

MTP

Multiscale Modeling & Simulation, 14(3):1153–1173, 2016

ACE

Physical Review B, 99(1):014104, 2019.

DeePMD Notes

1. A good starting point is to run the water examples. There are JSON input files available for various descriptors.

📦 https://github.com/deepmodeling/deepmd-kit/tree/master/examples/water

se_e2_a should be one of the standard descriptors to try. It is short for the Deep Potential Smooth Edition (DeepPot-SE), which is constructed from the angular and radial features of atomic configurations. Later model frameworks, such as dpa-1, 2, 3, stand for Deep Potential models with an attention mechanism using the descriptor se_atten.

2. sel is an important parameter, which means the expected maximum number of the neighbors of a type of atom. DeePMD-kit will allocate memory according to sel.

📖 https://docs.deepmodeling.com/projects/deepmd/en/stable/model/sel.html

dp --pt neighbor-stat -s data -r 6.0 -t O H

3. There is no available option to save the best model from the validation performance currently, Related posts as following:

💬 How to save the best model on validation data during model training? (Unanswered)

💬 Save model with best validation Error

There is a training parameter max_ckpt_keep can change the latest maximum number of the checkpoints to keep. We can adjust this larger and check the learning curve output data to choose the best model from validation performance metrics.

4. dpdata can be used to perform the conversion and generate the npy data, which is the recommended format in DeepMD, with the following command:

dpdata OUTCAR -i vasp/outcar -o deepmd/npy -O deepmd_data

If the training input is from single point energy calculations, there is one post from the discussion forum, but it seems they did not give a clear reply.

💬 DeepMD training model with Single Point energy calculations

We can simply concatenate the OUTCAR files (e.g., OUTCAR_1, OUTCAR_2, …) into a single OUTCAR and run this command to produce a dataset as if dealing with the AIMD OUTCAR.

However, it assumes that all images have the same atom types and counts (for example, AIMD OUTCAR outputs). This may not be valid for cases where the number and order of elements vary across different images. In that case, we should convert one-by-one with dpdata on the single point energy OUTCAR. The following is a related to-do enhancement from the deepmd-kit:

💬 Support batched inference with different natoms

5. There is a provided script to plot the learning curve log data from their documentation.

📖 https://tutorials.deepmodeling.com/en/latest/Tutorials/DeePMD-kit/learnDoc/Handson-Tutorial%28v2.0.3%29.html#train-process

6. There is no limit on the radius cutoff to be less than half box size, which is explained as following:

📖 https://docs.deepmodeling.com/projects/deepmd/en/stable/troubleshooting/howtoset_rcut.html