ZeoNet How-tos

author

2024-03-11 17:04 (updated 2025-10-06 16:35)

Comments

Tutorials

Introduction to ZeoNet (slides SharePoint folder, paper, GitLab repository), a representation learning framework using convolutional neural networks and 3D volumetric representations for predicting adsorption Henry in more than 330k zeolites.
Documentation to PyTorch, the main library we used for deep learning tasks.
Tutorials to Pymatgen, a robust python library for materials analysis.

Running jobs on HPC systems using Shotgun

Setup on group server

Set up shotgun on group server.
Complete Step 1 in the Shotgun workflow. For our ZeoNet project, the local_job_root is $HOME/projects/zeonet. The local_asset_dir is $HOME/projects/zeonet/bin. In the local_job_root, clone the ZeoNet repository from the GitLab repository to the local_asset_dir.
```
mkdir -p $HOME/projects/zeonet && cd $HOME/projects/zeonet
shotgun new-project  # and edit shotgun.yaml
git clone git@gitlab.com:baigroup/zeonet.git bin
rm -r bin/ML-Zeolites # remove sample dataset files
```
Create symbolic links for the dataset from the shared group directory.
```
ln -sf /home/user/NOBAK_projects/ML-Zeolites .
```
Upload the local_asset_dir to the HPC. Different HPCs have their own file systems, and we first need to figure out an appropriate filesystem location to be used as the primary location for running HPC jobs. Take Unity as an example, we first SSH to Unity to create a Scratch Workspace (the command below will create /scratch/workspace/user_umass_edu-ABC) to host the work directory. Edit shotgun.yaml to specify this work directory (e.g., /scratch/workspace/user_umass_edu-ABC/zeonet) as job_root.
```
ssh unity ws_allocate ABC  # For Unity only
shotgun upload-asset -t unity-24C-8G
```

Setup on HPC

We need to verify dataset including CIF structure files, volumetric grids HDF5 files, dataset split TXT files, and the zeolite property CSV file existing on HPC. On HPC systems where directories can be shared with the group, the target files usually already existed (e.g., /work/pi_pengbai_umass_edu/ML-Zeolites exist on unity) and, if not, we need to rsync everything ourselves (make sure to read step 4 of Shotgun basic workflow and please contact administrator then.

SSH to the HPC. Install Conda and create a new Conda environment from the environment-linux64-cuda121.txt file.

ssh unity
cd /scratch/workspace/$USER-ABC/zeonet # For Unity
module load conda/latest  # The conda version may be different for other HPCs
module load cuda/12.1
conda env create -f environment-linux64-cuda121.yml

The following steps must be done once. Since the dataset was already put in the shared group directory (e.g., /work/pi_pengbai_umass_edu/ML-Zeolites/), we only need to create symbolic links.
```
ln -sf /work/pi_pengbai_umass_edu/ML-Zeolites .
```

Submitting jobs

Prepare the job submission script. The best practice is to write a generic submission script and put it in the local asset directory (e.g., $HOME/projects/zeonet/bin), and then create symbolic links in each job directory:
```
ln -sf ../bin/sub-zeonet.script .  # Run in current directory, e.g., $HOME/projects/zeonet/BatchSize16-LR0.001-OptAdam
```
Create a config.yaml file. Refer to the configuration files in configs folder.

Submit the job using Shotgun.

shotgun submit -c 1 -g 1 -n 1 -t 336 -e 20G -r unity-24C-8G -j ../bin/sub-zeonet-unity.script

Running an interactive job on HPC systems

With an interactive job, we request time and resources to work on a compute node directly. In this way, we can interact with applications, execute scripts, or run other commands in real time within an HPC environment, which are convenient for debugging, designing workflows, and doing small tests. First SSH to the HPC and then request an interactive job.
```
ssh unity
srun --nodes=1 --ntasks-per-node=1 --cpus-per-task=1 --gpus=1 time=1:00:00 --job-name ML --partition ials-gpu --pty bash  # For Unity
```