Software¶

Language environments, machine learning frameworks, containers, and applications on CARC systems.

Python & Jupyter¶

• Conda and Anaconda: introduction - What conda is, how environments work, and how to use Anaconda/Miniconda on CARC systems.
• Managing conda environments - Create, activate, export, and remove conda environments on CARC clusters.
• Conda channels and pip - Use conda channels (conda-forge, bioconda) and mix pip installs safely inside environments.
• Conda environments in JupyterHub - Make your conda environments available as kernels in CARC JupyterHub.
• Installing deep learning packages - Install GPU-enabled deep learning frameworks (PyTorch, TensorFlow) into conda environments.
• Parallel Python with Dask and scikit-learn - Scale scikit-learn workloads across cluster nodes from JupyterHub using Dask.
• MPI parallelization from JupyterHub - Run MPI-parallel Python (mpi4py/ipyparallel) from CARC JupyterHub sessions.

R¶

• R on CARC systems - Load R, run scripts in batch jobs, and use R interactively on CARC clusters.
• Getting R software - Available R versions and how to load them with environment modules.
• Installing R packages - Install R packages into your user library on CARC systems.
• Parallel R with the future package - Parallelize R code across cores and nodes using the future framework.
• Gurobi optimizer with R - Use the Gurobi optimization solver from R on CARC clusters.
• R batch jobs with PBS (retired) - Historical instructions for submitting R jobs with PBS/Torque, which CARC has replaced with Slurm. (legacy)

MATLAB¶

• Running MATLAB jobs - Run MATLAB non-interactively in Slurm batch jobs on CARC clusters.
• Parallel MATLAB: profile setup and batch submission - Configure a cluster profile and submit parallel MATLAB jobs.
• MATLAB Parallel Server - Use MATLAB Parallel Server to scale parpool jobs across multiple nodes.
• MATLAB on GPUs - Accelerate MATLAB computations with GPUs on CARC clusters.
• MATLAB deep learning - Train deep learning models in MATLAB using CARC GPU nodes.

AI & machine learning¶

• PyTorch on CARC GPUs - Install and run GPU-enabled PyTorch on CARC clusters.
• PyTorch image classifier walkthrough - End-to-end example: train an image classifier with PyTorch on a CARC GPU node.
• TensorFlow on CARC GPUs - Install and run GPU-enabled TensorFlow on CARC clusters.
• Multi-GPU TensorFlow - Distribute TensorFlow training across multiple GPUs on a CARC node.
• AlphaFold - Run AlphaFold protein structure prediction on CARC systems.

Containers & tools¶

• Singularity / Apptainer containers - Build, pull, and run software containers on CARC clusters.
• Apache Spark - Launch Apache Spark clusters inside Slurm allocations for large-scale data analysis.
• ParaView remote visualization - Run the ParaView server on CARC compute nodes and connect from your desktop client.
• CUDA-aware MPI - Pass GPU device pointers directly to MPI calls with the CUDA-aware OpenMPI/UCX stack, and fix the mixed-environment segfault.