Coderefinery Scientific Computing Workshop Summer 2022

TODO: Fill in section from data storage to your science.

TODO: Fill in cooking analogy for parallel computing.

Installing things in a cluster environment is a bit different than installing them on a personal machine. More details are listed in this webpage. Assuming that we are using conda, we can first get conda by using module load miniconda in the terminal.

By default the installed packages and environments are in your home directory, which normally has a lower quota so it is a good idea to install packages and environments into your work directory. To do this, use

mkdir $WKRDIR/.conda_pkgs
mkdir $WRKDIR/.conda_envs

conda config --append pkgs_dirs ~/.conda/pkgs
conda config --append envs_dirs ~/.conda/envs
conda config --prepend pkgs_dirs $WRKDIR/.conda_pkgs
conda config --prepend envs_dirs $WRKDIR/.conda_envs

It is good practice to write an environment.yml file that describes the environment rather than installing environments from the command line. An example environment.yml file is given below.

name: conda-example
channels:
  - conda-forge
dependencies:
  - numpy
  - pandas

To create this environment, use:

module load miniconda
conda env create --file environment.yml

Then we can activate the environment via conda activate conda-example.

For a crash course on the Linux shell, see here. For details on how to connect to Triton (Aalto University cluster) see here.