Scrapbook (Posts about coding)

Compilation & Tuning

author — Sat, 19 Mar 2022 04:48:10 GMT

HPC machine microarchitecture

ACCESS NCSA
- Delta (AVX2): 2.45GHz 128 core, 2GB/core: 2 x AMD EPYC 7763 "Milan" (Zen 3, 32MB L3), 256GB
- Delta-GPU (AVX2; cc86): 4 x NVIDIA A40 (48Gb), 2.45GHz 64 core, 4GB/core: AMD EPYC 7763 "Milan", 256GB
- Delta-GPU (AVX2; cc80): 4 x NVIDIA A100 (40Gb), 2.45GHz 64 core, 4GB/core: AMD EPYC 7763 "Milan", 256GB
- Delta-GPU (AVX2; cc80): 8 x NVIDIA A100 (40Gb), 2.45GHz 128 core, 16GB/core: 2 x AMD EPYC 7763 "Milan", 2048GB
- Delta-GPU (AVX2; CDNA): 8 x AMD MI100 (32Gb), 2.45GHz 128 core, 16GB/core: 2 x AMD EPYC 7763 "Milan", 2048GB
ACCESS JHU
- Rockfish (AVX512): 3.0GHz 48 core, 4GB/core: 2 x Intel Xeon 6248R (Cascade Lake), 192GB DDR4
- Rockfish-GPU (AVX512): 4 x NVIDIA A110 (40Gb), 3.0GHz 48 core, 4GB/core: 2 x Intel Xeon 6248R (Cascade Lake), 192GB DDR4
ACCESS Purdue
- Anvil (AVX2): 2.45-3.5GHz 128 core, 2GB/core: 2 x AMD EPYC 7763 (Zen 3), 256GB DDR4, 56 Gb/s Interconnect
- Anvil-GPU (AVX2; cc80): 4 x NVIDIA A100, 2.45-3.5GHz 128 core, 4GB/core: 2 x AMD EPYC 7763 (Zen 3), 512GB DDR4, 56 Gb/s Interconnect
ACCESS PSC
- Bridges2 (AVX2): 2.25-3.40GHz 128 core, 2GB/core: 2 x AMD EPYC 7742 (Zen 2, 256MB L3), 256GB DDR4, Mellanox ConnectX-6-HDR Infiniband 200Gb/s Adapter
- Bridges2-GPU (AVX512; cc70): 8 x NVIDIA Tesla V100 32GB SXM2, 2.5-3.9GHz 40 core, 12.8GB/core: 2 x Intel Xeon Gold 6248 (Cascade Lake), 512GB DDR4, Mellanox ConnectX-6-HDR Infiniband 200Gb/s Adapter
- Bridges2-AI-1 (AVX512; cc70): 8 x NVIDIA V100 16GB, 2.4-3.7GHz 40 core, 4.8GB/core: 2 x Intel Xeon Gold 6148 (Skylake), 192GB DDR4, Mellanox ConnectX-6-HDR Infiniband 200Gb/s Adapter
- Bridges2-AI-2 (AVX512; cc70): 16 x NVIDIA Volta V100 32GB, 2.7-3.7GHz 48 core, 32GB/core: 2 x Intel Xeon Platinum 8168 (Skylake), 1.5TB DDR4, Mellanox ConnectX-6-HDR Infiniband 200Gb/s Adapter
- Bridges (AVX2): 2.3/3.3GHz 28 core, 4.57GB/core: 2 x Intel Xeon E5-2695v3 (Haswell, 35MB LLC), 128GB DDR4
ACCESS SDSC
- Expanse (AVX2): 2.25-3.40GHz 128 core, 2GB/core: 2 x AMD EPYC 7742 (Zen 2, 256MB L3), 256GB DDR4, 56 Gb/s Interconnect
- Expanse-GPU (AVX512; cc70): 4 x NVIDIA V100 SMX2, 2.5-3.9GHz 40 core, 9.6GB/core: 2 x Intel Xeon Gold 6248 (Cascade Lake), 384GB DDR4
- Comet (AVX2): 2.5GHz 24 core, 5.33GB/core: 2 x Intel Xeon E5-2680v3 (Haswell), 128GB DDR4, 56Gbps FDR InfiniBand with rack-level full bisection bandwidth and 4:1 oversubscription cross-rack bandwidth
ACCESS UDel
- Darwin (AVX2): 2.5-3.35GHz 64 core, 8GB/core: 2 x AMD EPYC 7502 (Zen 2, 128MB L3), 512GB, Mellanox HDR100
- Darwin-GPU: NVIDIA T4 (cc75) / NVIDIA V100 (cc70) / AMD MI50
ACCESS UTexas
- Stampede3
  - SPR (AVX512): 1.90-3.50GHz 112 core 1.14GB/core: 2 x Intel Xeon CPU MAX 9480 ("Sapphire Rapids HBM, 48KB L1 + 1MB L2 + 112.5MB L3 per socket), 128 GB HBM 2e, 100Gb/sec Intel Omni-Path (OPA) network (interconnected with PVC nodes)
  - PVC (AVX512; Xe-HPC): CPU: 2.10-3.70GHz 112 core 4.57GB/core: 2 x Xeon CPU MAX 8480 (Sapphire Rapids, 48KB L1 + 1MB L2 + 112.5MB L3 per socket), 512 HB DDR5. GPU: 900-1600 MHz 512 core 1GB/core: 4 x Intel Data Center GPU Max 1550s, 100Gb/sec Intel Omni-Path (OPA) network (interconnected with SPR nodes)
  - ICX (AVX512): 2.3GHz 80 core 3.2GB/core: 2 x Intel Xeon Platinum 8380 (IceLake, 48KB L1 + 1.25MB L2 + 60MB L3 per socket), 256GB DDR4 3.2GHz, 100Gb/sec Intel Omni-Path (OPA) network
  - SKX (AVX512): 2.1GHz 48 core 4GB/core: 2 x Intel Xeon Platinum 8160 (Skylake, 32KB L1 + 1MB L2 + 33MB L3 per socket), 192GB DDR4 2.67GHz, 100Gb/sec Intel Omni-Path (OPA) network
- Stampede2 (multi-architecture: -xCOMMON-AVX512 -axCORE-AVX512,MIC-AVX512)
  - KNL (MIC-AVX512): 1.4GHz 68 core 1.4GB/core: Intel Xeon Phi 7250 (Knights Landing, 32KB L1 + 1MB L2 per two core + 16GB MCDRAM), 96GB DDR4 2400MHz
NERSC
- Perlmutter (AVX2): 2.45-3.5GHz 128 core, 4GB/core: 2 x AMD EPYC 7763 (Zen 3), 512GB DDR4, Slingshot 11 Interconnect
- Perlmutter-GPU (AVX2; cc80): 4 x NVIDIA A100, 2.45-3.5GHz 64 core, 4GB/core: AMD EPYC 7763 (Zen 3), 256GB DDR4, Slingshot 11 Interconnect
- Cori KNL (MIC-AVX512): 2.3GHz 32 core 4GB/core: 2 x Intel Xeon E5-2698v3 (Haswell, 32KB L1 + 256 KB L2 + 40MB L3 per socket), 128 GB DDR4 2133 MHz, Cray Aries with Dragonfly topology
- Cori (AVX2): 1.4GHz 68 core 1.4GB/core: Intel Xeon Phi 7250 (Knights Landing, 32KB L1 + 1MB L2 per two core + 16GB MCDRAM), 96 GB DDR4 2400 MHz
- Edison (AVX): 2.4GHz 24 core, 2.67GB/core: 2 x Intel E5-2695v2 (Ivy Bridge), 64GB DDR3 1866MHz, Cray Aries dragonfly topology
ANL
- Mira: IBM Blue Gene/Q
UMass
- Unity (multi-architecture: -xCORE-AVX2 -axCORE-AVX512)
  - Intel Skylake
  - AMD Zen 2
  - GPU: GTX Titan X (cc52), GTX 1080ti (cc61), RTX 2080 (cc75), RTX 2080ti (cc75), Quadro RTX 8000 (cc75), Tesla M40 (cc52), Tesla V100 (cc70), Tesla A100 (cc80)
- GHPCC
  - Intel Skylake -> AVX512
  - Haswell -> use Mesabi builds
  - Sandy Bridge -> Itasca_sb builds
  - AMD Piledriver, Bulldozer -> Itasca Nehalem builds
Others
- Enterprise (AVX2): 2.2GHz 24 core, 2.67GB/core: 2 x Intel Xeon E5-2650v4 (Broadwell), 64GB DDR4
- Mesabi (AVX2): 2.5GHz 24 core, 2.67GB/core: 2 x Intel Xeon E5-2680v3 (Haswell), 64GB DDR4
- Metropolis (AVX)
  - 2.4GHz 8 core, 2GB/core: 2 x Intel Xeon E5-2609 (Sandy Bridge), 16GB
  - 2.5GHz 8 core, 2GB/core: 2 x Intel Xeon E5-2609v2 (Ivy Bridge), 16GB
  - 2.1GHz 8 core, 2GB/core: 2 x Intel Xeon E5-2620v2 (Ivy Bridge), 16GB
  - 2.1GHz 16 core: 2 x Intel Xeon E5-2620v4 (Broadwell)
- Itasca
  - _sb suffix (AVX): Intel Sandy Bridge
  - Regular (SSE4.1): Intel Nehalem
- 4170HE: AMD K10

Compiler flags

CP2K, 7.1 H20-256

AOCC+AOCL: -O3 -march=znver3 -ffast-math -fopenmp -ffree-form -Hx,47,0x10000008 -Mbackslash

VASP, 6.1.2 GaAsBi-64 (VASP guide refers to -02 and not -O3 as default optimisation level)

Intel: -O2 -assume byterecl -w -march=core-avx2 -fma -mavx2 -mfma -free -names lowercase -mkl=sequential -qopenmp -lmkl_blacs_openmpi_lp64 -lmkl_scalapack_lp64

Quantum Espresso, 6.5 AUSURF112

GCC+AOCL: -O3 -march=znver2 -ffast-math -fopenmp

NAMD, 2.14

AOCC+AOCL: -optimize -production -DCMK_OPTIMIZE -O3 -Ofast -g -m64 -march=znver3 -fPIC -ffp-contract=fast -fopenmp -Wno-deprecated-register -Wno-non-c-typedef-for-linkage -mllvm -inline-threshold=3000 -finline-hint-functions -mllvm -inlinehint-threshold=10000 -finline-functions

WRF, 3.9.1.1, CONUS 2.5KM

AOCC: -march=znver3 -ffp-contract=fast -mavx2 -mfma -mprefer-vector-width=256 -Ofast -ffast-math -fopenmp -m64 -w -finline-aggressive -fPIC

OpenFOAM, 2012, Motorbike 42m cells

GCC: -O3 -march=znver2 -mfma -ffp-contract=fast -funroll-loops -fno-signed-zeros -fno-trapping-math -fno-math-errno -ffinite-math-only -fno-rounding-math -fno-signaling-nans -fcx-limited-range -fexcess-precision=fast

Generate auto-dispatch code paths with Intel compilers

Some HPC systems (e.g., NERSC Cori, XSEDE Stampede2) have different kinds of compute nodes with different instruction sets: Broadwell (-xCORE-AVX2), Skylake/Icelake (-xCORE-AVX512, and KNL (-xMIC-AVX512).

A simpler approach can be used when building with the Intel compiler: use CPU auto dispatch to build a multi-architecture ("fat") binary that contains alternate code paths with optimized vector code for each type of compute nodes. To produce a multi-architecture binary for Haswell, Skylake/Icelake, and KNL nodes, build with the following options -xCORE-AVX2 -axCORE-AVX512,MIC-AVX512.

These particular choices allow you to build on any node and use CPU dispatch to produce a multi-architecture binary. We recommend that you specify these flags in both the compile and link steps. Specify an optimization level (e.g. -O3) along with any other appropriate compiler switches: icc -xCORE-AVX2 -axCORE-AVX512,MIC-AVX512 -O3 mycode.c -o myexe.

The -x option is the target base architecture (instruction set). The base instruction set must run on all targeted processors. Here we specify CORE-AVX2, which is native for older Broadwell processors and supported on all KNL and Skylake/Icelake nodes. This option allows configure scripts and similar build systems to run test executables on any login or compute node. The -ax option is a comma-separated list of alternate instruction sets: CORE-AVX512 for Skylake and Icelake, and MIC-AVX512 for KNL.

If compatibility with Broadwell nodes is not needed, more aggressive base option is -xCOMMON-AVX512, which is a subset of AVX512 that runs on all KNL, Skylate, and Icelake nodes.

On heterogeneous HPC clusters, it's best to avoid building with -xHost (a flag that means "optimize for the architecture on which I'm compiling now"). Using -xHost on a Skylake node, for example, will result in a binary that won't run on KNL.

Don't skip the -x flag in a multi-architecture build: the default is the very old SSE2 (Pentium 4) instruction set. Don't create a multi-architecture build with a base option of either -xMIC-AVX512 (native on KNL) or -xCORE-AVX512 (native on Skylake/Icelake); there are no meaningful, compatible alternate (-ax) instruction sets:

$ icc -xCORE-AVX512 -axMIC-AVX512 -O3 mycode.c -o myexe  # WRONG! Base incompatible with alternate

Simplicity is a major advantage of this multi-architecture approach: it allows you to build and run on any nodes, and performance is generally comparable to single-architecture builds. There are some trade-offs to consider, however. This approach will take a little longer to compile than single-architecture builds, and will produce a larger binary. In some cases, you might also pay a small performance penalty over single-architecture approaches. For more information see the Intel documentation.

Auto-dispatch for both Intel and AMD CPUs

For HPC systems (e.g., Unity) with both AMD and Intel compute nodes, we may build auto-dispatch executables with -march=core-avx2 -mtune=core-avx2 -axCORE-AVX512. See the notes below for more details.

Intel compilers for AMD

The Intel compiler is developed and targeted for the Intel hardware and hence it has some minor issues when using it with AMD hardware. Execution of code compiled with the Intel compiler might lead to crash when executed on the AMD EPYC CPUs. The recommended flag to use the AVX2 instructions supported by the AMD EPYC CPUs is -march=core-avx2 -mtune=core-avx2.

In some cases, when compiling with the -xavx, -xavx2 or -xcore-avx2 flags your application will abort with a message that the processor is not compatible:

Please verify that both the operating system and the proce_ssor support Intel(R) X87, CMOV, MMX, FXSAVE, SSE, SSE2, SSE3, SSSE3, SSE4_1, SSE4_2, MOVBE, POPCNT, AVX, F16C, FMA, BMI, LZCNT and AVX2 instructions.

To solve this problem, avoid using the above-mentioned flags for the compilation of the main() function. Avoiding the use of Interprocedural Optimization (remove the -ipo flag) may also be tried.

Building notes

NOTE: These are abbreviated notes that have worked in the past. Always read the current documentation, and only use these notes as hints and examples for putting together a working build.

LAMMPS

Official Documentation

CMake procedure

module add intel/2015/update3; module add impi/5.0.2.044/intel
mkdir ~/tools/lammps-git/build
cd ~/tools/lammps-git/build
cmake -DCMAKE_CXX_COMPILER=mpiicpc -DCMAKE_C_COMPILER=mpiicc -DCMAKE_Fortran_COMPILER=mpiifort -DCMAKE_TUNE_FLAGS='-O3 -xHost -ipo -fp-model fast=2 -no-prec-div -qoverride-limits -diag-disable=10441 -diag-disable=2196' -DBUILD_MPI=yes -DBUILD_OMP=yes -DFFT=MKL -DLAMMPS_MEMALIGN=64 -DBUILD_SHARED_LIBS=on -DLAMMPS_EXCEPTIONS=yes -DPKG_COLVARS=on -DPKG_INTEL=on -DPKG_LEPTON=on -DPKG_KOKKOS=on -DPKG_KSPACE=yes -DPKG_MOLECULE=yes -DPKG_EXTRA-MOLECULE=on -DPKG_OPENMP=on -DPKG_OPT=on -DPKG_PYTHON=on -DPKG_REAXFF=on -DPKG_REPLICA=on -DPKG_RIGID=on -DCMAKE_INSTALL_PREFIX=$HOME/tools/lammps/intel-impi-mkl ../cmake
make -j 2

Example cmake line for GCC+

cmake -DCMAKE_CXX_COMPILER=mpicxx -DCMAKE_C_COMPILER=mpicc -DCMAKE_Fortran_COMPILER=mpif90 -DCMAKE_TUNE_FLAGS='-O3 -mtune=native -march=native -ffast-math -fassociative-math -freciprocal-math -fno-signed-zeros -fno-trapping-math -funroll-loops' -DBUILD_MPI=yes -DBUILD_OMP=yes -DFFT=FFTW3 -DLAMMPS_MEMALIGN=64 -DBUILD_SHARED_LIBS=on -DLAMMPS_EXCEPTIONS=yes -DPKG_COLVARS=on -DPKG_INTEL=on -DPKG_LEPTON=on -DPKG_KOKKOS=on -DPKG_KSPACE=yes -DPKG_MOLECULE=yes -DPKG_EXTRA-MOLECULE=on -DPKG_OPENMP=on -DPKG_OPT=on -DPKG_PYTHON=on -DPKG_REAXFF=on -DPKG_REPLICA=on -DPKG_RIGID=on -DCMAKE_INSTALL_PREFIX=$HOME/tools/lammps/gcc-mpich2-fftw3 ../cmake

Reax (old Make system)

cd ~/tools/lammps-git/lib/reax; make -f Makefile.ifort # build Fortran reax library
cd ../../src
# Modify CCFLAGS, LINKFLAGS, FFT_INC, and FFT_LIB in MAKE/Makefile.ompi_icc to add optimizations and point to MKL libraries
# To use FFTW3: set FFT_INC = -DFFT_FFTW3, FFT_LIB = -lfftw3, and do module load fftw/3.3-ompi-double
make yes-opt yes-reax yes-user-intel yes-user-omp yes-user-reaxc
make yes-kspace yes-manybody yes-molecule yes-opt yes-reax yes-replica yes-user-intel yes-user-omp yes-user-reaxc
make ompi_icc -j 16

AOCC+AOCL 29-OCT-20 lj

-g -O3 -march=znver2 -ffast-math -ffp-contract=fast -DNDEBG -pthread -fopenmp -std=c++11

GROMACS

Official documentation: Do not rely solely on the "Quick and dirty installation". You most likely need to read through the entire Prerequisites section to build an executable with decent performance.

Example procedure

gcc-mvapich-fftw

mkdir ~/tools/gromacs-5.0.4/build
cd ~/tools/gromacs-5.0.4/build
module add gcc mvapich
cmake -DCMAKE_C_COMPILER=mpigcc -DCMAKE_CXX_COMPILER=mpigxx -DGMX_DOUBLE=ON -DGMX_MPI=ON -DGMX_FFT_LIBRARY=fftw3 -DGMX_BUILD_OWN_FFTW=ON -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-2016.1/gcc-double-mpi ..
make -j 16 install

intel-impi-mkl

module add intel/2013/update2 impi/4.1.1.036/intel mkl/11.1.1.106
cmake -DCMAKE_C_COMPILER=mpiicc -DCMAKE_CXX_COMPILER=mpiicpc -DGMX_DOUBLE=ON -DGMX_MPI=ON -DGMX_FFT_LIBRARY=mkl -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-5.0.4/build ..

intel-ompi-fftw

module add intel/2013/update2 ompi/1.6.3/intel-2013-update2
cmake -DCMAKE_C_COMPILER=mpiicc -DCMAKE_CXX_COMPILER=mpiicpc -DGMX_DOUBLE=ON -DGMX_MPI=ON -DGMX_FFT_LIBRARY=fftw3 -DGMX_BUILD_OWN_FFTW=ON -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-5.0.4/build ..

gcc-ompi-fftw-cuda (single-precision only for GPU)

module add gcc/4.8.1 ompi/1.7.2/gnu-4.8.1 cuda/6.0
cmake -DCMAKE_C_COMPILER=mpigcc -DCMAKE_CXX_COMPILER=mpigxx -DGMX_MPI=ON -DGMX_GPU=on -DGMX_FFT_LIBRARY=fftw3 -DGMX_BUILD_OWN_FFTW=ON -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-5.0.4/build ..
make -j 16 install

Cray system with Intel KNL MIC (static build)

mkdir ~/tools/gromacs-2019.3/build
cd ~/tools/gromacs-2019.3/build
cmake -DCMAKE_C_COMPILER=cc -DCMAKE_CXX_COMPILER=CC -DGMX_DOUBLE=ON -DGMX_MPI=ON -DGMX_FFT_LIBRARY=mkl -DGMX_BUILD_SHARED_EXE=off -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-2019.3/intel-double-impi-mkl ..
make -j 16 install
rm -r *
cmake -DCMAKE_C_COMPILER=cc -DCMAKE_CXX_COMPILER=CC -DGMX_DOUBLE=ON -DGMX_MPI=ON -DGMX_FFT_LIBRARY=mkl -DGMX_BUILD_SHARED_EXE=off -DGMX_SIMD=AVX_512_KNL -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-2019.3/intel-double-impi-mkl-knl ..
module swap craype-haswell craype-mic-knl
make -j 16 install

Specifying prebuilt FFTW

module load fftw/3.3-ompi-double
CC=mpicc CXX=mpiCC FFTW3_ROOT_DIR=/soft/fftw/intel-ompi/3.3-double cmake -DGMX_DOUBLE=ON -DGMX_MPI=ON -DGMX_PREFER_STATIC_LIBS=ON -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-4.5.5/build ..

Specifying MKL libraries (typically not needed)

export MKL=/soft/intel/mkl/10.2.7.041
CC=mpicc CXX=mpiCC cmake -DGMX_DOUBLE=ON -DGMX_MPI=ON -DGMX_THREADS=OFF -DGMX_PREFER_STATIC_LIBS=ON -DCMAKE_INSTALL_PREFIX=$HOME/tools/gromacs-4.5.5/build -DGMX_FFT_LIBRARY=mkl -DMKL_INCLUDE_DIR=${MKL}/include "-DMKL_LIBRARIES=${MKL}/lib/em64t/libmkl_intel_lp64.so;${MKL}/lib/em64t/libmkl_core.so;${MKL}/lib/em64t/libmkl_sequential.so" ..
make install -j 8

AOCC+AOCL 2020.4 water_1536

-DGMX_DOUBLE=0 -O3 -Ofast -g -m64 -march=znver3 -fPIC -ffast-math -ffp-contract=fast -Wno-deprecated-copy -Wno-error=deprecated-copy --gcc-toolchain=/usr -O3 -DNDEBUG -mavx2 -mfma -funroll-all-loops -pthread -fopenmp=libomp

Post-processing/Analysis

author — Mon, 05 Dec 2016 23:29:14 GMT

Matlab

A utility function for processing variable length input argument list

% procVarArg Utility function for processing variable length input argument
% list
%
% Syntax
%   options = procVarArg(names, defaultValues, varArg)
%   [options, varargout] = procVarArg(names, defaultValues, varArg)
%
% Description
%   options = procVarArg(names, defaultValues, varArg) generates a comma
%   separated list and call struct, options, with k attribute names defined
%   in names (k by 1 matrix) and k default values defined in defaultValues
%   (k by 1 matrix). The property/value pairs that are explictly given in
%   varArg overwrite those in defaultValues. varArg is usually the variable
%   length input argument list, varargin, to the function calling
%   procVarArg.
%
%   [options, varargout] = procVarArg(names, defaultValues, varArg) stores
%   parameters from varArg that are not found in names in varargout.
%   Currently, vargargout support only one variable that will store all the
%   parameters not in names.
%
% See also varargin varargout

function [options,varargout] = procVarArg(names,defaultValues,varArg)

structInput=cell(2,length(names));
structInput(1,:)=names';
structInput(2,:)=defaultValues';
options=struct(structInput{:});

nVArgOut=max(nargout,1)-1;
if nVArgOut>1
    error('ternsurf:TooManyOutputVar','Too many output variables. Currently support at most two output variables.');
end
iVarArgOut=0;

nVarArg=size(varArg,2);
iVarArg=1;
flag=2;

while iVarArg<=nVarArg
    arg=varArg{iVarArg};
    if flag==1
        if ischar(arg)
            arg=lower(deblank(arg));
        end
        options.(names{iName,:})=arg;
        flag=2;
    elseif flag==2
        if ~ischar(arg)
           error('ternsurf:ParamNotString','Parameter name must be a string.',iVarArg);
        end
        iName=find(strncmpi(arg,names,length(arg)));
        if isempty(iName)
            flag=0;
            continue
        elseif length(iName)>1
            jName=find(strcmpi(arg,names));
            if length(jName)==1
                iName=jName;
            else
                allMatchedNames=['(',names{iName(1),:}];
                for jName=iName(2:length(iName))'
                    allMatchedNames=[allMatchedNames,', ',names{jName,:}];
                end
                allMatchedNames=[allMatchedNames,')'];
                error('ternsurf:AmbiguousParamName','Ambiguous parameter name: ',arg,allMatchedNames);
            end
        end
        flag=1;
    else
        iVarArgOut=iVarArgOut+1;
        varargout{iVarArgOut}=varArg{iVarArg};
    end
    iVarArg=iVarArg+1;
end

Default plot styles

Name the following file startup.m and put it in a directory in the MATLAB search paths.

% startup.m
%
% Set default values of several properties for the graphic object with
% handle 0, which is the root ancestor and default behavior for all plots.
% PropertyName is formed by concatenating the string 'Default', the object
% type (e.g., 'Figure', 'Axes', 'Line', and 'Text'), and the property name
% (e.g., 'Color', 'FontName', 'LineWidth').
%
% See also STARTUP SET

set(0, 'DefaultFigureColor', 'white', ...
'DefaultAxesColor', 'none', ...
'DefaultAxesFontAngle', 'normal', ...
'DefaultAxesFontName', 'Helvetica', ...
'DefaultAxesFontUnits', 'points', ...
'DefaultAxesFontSize', 24, ...
'DefaultAxesFontWeight', 'normal', ...
'DefaultAxesGridLineStyle', ':', ...
'DefaultAxesLayer', 'Bottom', ...
'DefaultAxesLineWidth', 2, ...
'DefaultAxesTickLength', [0.02,0.02], ...
'DefaultAxesUnits', 'inches', ...
'DefaultAxesPosition',[0.85 0.8 3.25 3.25],...
'DefaultAxesVisible', 'on', ...
'DefaultAxesXcolor', 'black', ...
'DefaultAxesYcolor', 'black', ...
'DefaultAxesZcolor', 'black', ...
'DefaultLineColor', 'black', ...
'DefaultLineLineStyle', '-', ...
'DefaultLineLineWidth', 2, ...
'DefaultLineMarker', 'none', ...
'DefaultLineMarkerSize', 9, ...
'DefaultTextColor', 'black', ...
'DefaultTextFontName', 'Helvetica', ...
'DefaultTextFontUnits', 'points', ...
'DefaultTextFontSize', 24, ...
'DefaultTextFontWeight', 'normal', ...
'DefaultTextHorizontalAlignment', 'left', ...
'DefaultTextInterpreter','latex', ...
'DefaultTextVerticalAlignment', 'middle' ...
);

Coding Guidelines

author — Wed, 30 Jan 2013 19:22:13 GMT

Getting Started

Software Carpentry can help you learn basic coding practices: working with a Unix shell, using Git, and basic Python commands. There is also a nice overview by the NSF MolSSI group on scripting for scientific computing.
It pays to have a reasonable familiarity with a terminal editor such as Vim, which makes editing and working on remote machines easier.
Python: You should familiarize yourself with PEP 8 guidelines and use tools like Pylint, Google’s yapf, or Sublime Text’s AutoPEP8 to enforce good style. It can also be helpful to use optional type hints so that others can more easily understand your code. Google has a comprehensive style guide for Python code.
While working in Jupyter notebooks is a great way to analyze experiments and provide tutorials of code, rigorous testing and reproducible research is more easily conducted with a well-organized and documented codebase (see here for an example layout).
Git: Get started with this tutorial and Chapter 2: Git Basics from the Pro Git book. Check out various git commands on its man page. For more complete information, see git documentation, particularly the user manual and the Pro Git book. For a quick interactive git tutorial, check learn git branching.
For reproducibility, you should always use a dependency manager like Conda or, for more complex dependencies beyond Python packages, Docker containers (see here for a short tutorial).
If you’re not sure what your development workflow should be, or how to use certain tools like Git, please just ask the group! We all have to learn this for the first time at some point, so there is no shame in asking. If you think “there must be a better way to do this”, there probably is.
We should always try to look for a general solution than an ad-hoc fix. When problems we are dealing with grow in complexity (they inevitably will), we will find solutions that are general in nature and clear in logic are easier to extend and maintain.
If we find ourselves doing the same thing twice, odds are that there is a better way to it. Spend 10 minutes (even hours) to learn a technique that can be implemented in 1 minute, rather than spend 1 minute to come up with a technique that takes 10 minutes. Even if the learning phase takes longer, it will usually pay off in the long run: you may likely apply the better method many times in the future and you have sharpened your coding skills.

Using Git

Git Tutorials

This website is a useful tool for learning Git with simple, step-by-step tutorials and good visuals

Git Guidelines

Pull changes from the upstream repository frequently, so that conflicts can be dealt with easily. We often tend to defer this work to save time, but this debt is always paid later in much bigger proportions.
Merge to the upstream repository frequently (through a Pull Request), so other people can checkout your changes and stay sync.
Double check that changes to be submitted are correct and do not change other functionality inadvertently. Run the test suite if it exists.
Remove unnecessary files (any files that can be regenerated) before pushing, such as editor generated backups (*~) and temporary files (#*#), compiled executables (they are too large and should not be managed by version control systems), and simulation output from running test suites. Learn to use .gitignore files to ignore unwanted files.
Never commit any proprietary information (e.g., private data, passwords), even to a private repository. Triple check this before making any repository public.
Before merging with upstream, clean up the commit history to include only major, self-contained entries. Squash small, bookkeeping commits (e.g., test1, debug2, try3) or use the Squash and Merge functionality provided by GitLab.

Sample Git Workflow

This brief walk-through augments, but does not replace the tutorials suggested in Point 5 of Getting Started.
Install git on Ubuntu or WSL:

sudo apt update
sudo apt install git

Set up your git user name and email:

git config --global user.name "Full Name"
git config --global user.email you@university.edu

Clone a repository on GitLab from the group space to your user space.
Clone the repository to your laptop or group server over ssh:

git clone git@gitlab.com:USERNAME/MCCCS MCCCS

This will create a new directory called "MCCCS" under the current working directory, and clone (only) the "master" branch from the remote side.

Show the remote repositories that have been set up with your working tree:

git remote
git remote show origin

The remote repository "origin" is automatically created when you do "git clone".

Add the upstream repository from the group space:

git remote add upstream git@gitlab.com:group-server/MCCCS

Examine the branches currently available:

git branch # local branches, only having "master"
git branch -r # remote branches, containing "master" and "serial"
git branch -a # show both local and remote branches

"serial" is a branch that holds some coarse graining stuff. The "master" branch is the MPI parallel version. Remote branches are a mirror image of branches in remote repositories and you NEVER switch to them directly or write to them.

Suppose you want to work on the "serial" branch, DO NOT switch to them directly! Instead, create a corresponding local branch which will "track" the remote branch (remote-tracking branch):

git checkout -b serial origin/serial

You can now check the available branches again by "git branch -a".

Switch back to the "master" branch:

git checkout master

Make some changes.
Frequently you should commit your changes to your working tree:

git status # this will show any modified, added, and deleted files
git commit -a # commit all changes including those unstaged

This will only update your local repository.

If changes have been made in the upstream repository, we can update our working tree by:

git fetch -p upstream
git merge upstream/master

If you have been making changes locally that conflict with upstream, git will complain and you have to commit your changes to your working tree first. We will then use the rebase workflow to merge changes:

git rebase --onto upstream/master <LAST_COMMON_COMMIT> HEAD

where is the last commit common between upstream and your working tree. Make sure to check out the documentation and understand this command.

Next, when we are satisfied and our changes have passed the test suites, we can push them back to origin:

git push origin master

Create a a Pull Request on GitLab against the upstream repository.
Now suppose we are going to do something big, making a new branch could be a good idea:

git branch test-miracle  # new branch will be based on the currently checked out branch
git checkout test-miracle  # switch to the new branch

After the new feature has been fully tested, you can merge them back to the "master" branch. Make sure you are on one of the to-be-merged branches and merge the other one now:

git checkout master  # make sure you are on the to-be-merged branch
git merge test-miracle  # merge the other branch

If changes have been made on only "test-miracle" since the last merge, they are simply replayed on "master" (i.e., fast-forward merge). If changes have been made on both branches, they are merged intelligently (i.e., three-way merge). If there are conflicts, git merge will report them and let you resolve them, updating the rest of the tree already to the result state; you can git commit when you resolve the conflicts. If no changes conflicted, a commit is made automatically with a convenient log message (or you can do git merge --no-commit branch to review the merge result and then do the commit yourself).

Check the differences between two files, directories, or branches:

git diff

Browse the history of the development either in text or graphical mode:

git log  # show full history in plain text
git log serial..master  # show everything that is reachable from "master" but exclude anything that is reachable from "serial",
                        # i.e., show the changes between the current state of "master" and the point where the two branches diverge
git log serial...master  # show everything that is reachable from either one, but exclude anything that is reachable from both of them
gitk &  # show full history in graphical mode
gitk serial..master  & # same two-dot range notation as used with git log
gitk serial...master  & # same three-dot range notation as used with git log

Please refer to the user manual for specifics on these commands.

Installing Git

Mac OS X

You can install git from Fink:

fink install git

or MacPorts:

port install git-core

or brew:

brew install git

or using Git OSX Installer if you are running 10.5 or above.

Linux

On Fedora, CentOS, RedHat:

yum install git

On Ubuntu, Debian:

apt-get update
apt-get install git

Windows

Download and install the msysGit package.

Installing from source

cd ~
wget http://git-core.googlecode.com/files/git-1.7.9.tar.gz # Refer to [http://git-scm.com/download] for most current versions
tar xzf git-1.7.9.tar.gz
cd git-1.7.9
make prefix=/usr all
sudo make prefix=/usr install

Profiling Your Code

To properly speed-up your code's implementations, it's best to have a targeted approach. Rather than timing your code, making changes, and then retiming, it's best to know what portions of your code is taking the most time and make improvements to that function/subroutine. This is where profilers come in. Profilers will can analyze each function of your code and determine the time it takes or the memory required to run your code.

Language Agnostic

We can apply the Monte Carlo method for profiling our code. If we run our code and in random instances interrupt our program (ctrl-c) and determine which function our code was in, if there's a function that's particularly slow, we'll find that the code is being interrupted in that section of code a majority of the time. With enough samples, we can find the sore spots in the code and make the appropriate adjustments there. For more information, see this StackOverflow post
For cpu profiling, see VTune
For memory profiling, see valgrind for a multi-language solution

Python

For cpu profiling, cProfile is a built-in method for deterministically profiling your python code. After running the profiler, use the Stats class to analyze the data from the profiler. It will output the times each function is called, how long each function runs per call, the cumulative time it takes for each function to run, and much more.

For memory usage, memray provides good visualization of how memory is allocated in your script and can simply be installed using the pip package manager.

General Tips

When profiling your code, it may not be enough to compare a single variation of your code. Try making small changes to see what correlations you can glean (e.g. Run 100 frames of analysis and then 200 frames of analysis and see how memory/time scales. Does it scale linearly? Is this expected? Is memory that should be deallocated still allocated?)