All in One View

Content from What is OnDemand?

Last updated on 2026-08-27 | Edit this page

Estimated time: 20 minutes

Overview

Questions

  • What is OnDemand and why use it?
  • How does OnDemand differ from SSH?
  • What are the benefits for HPC users?

Objectives

  • Understand the purpose and benefits of OnDemand
  • Learn the difference between OnDemand web interface and command-line SSH access
  • Identify use cases where OnDemand is most valuable

Introduction to OnDemand

OnDemand is a web-based interface to access and control Sagehen HPC resources without requiring SSH knowledge or specialized terminal software. It provides a graphical dashboard through which you can manage files, submit jobs, monitor computations, and run interactive applications directly from your web browser.

Why Use OnDemand?

OnDemand provides several advantages over traditional SSH access:

  • No SSH knowledge required: Access HPC resources through a familiar web browser
  • Graphical file manager: Upload, download, and organize files without command-line tools
  • Interactive applications: Run Jupyter notebooks, RStudio, and other interactive apps directly
  • Job monitoring: Visualize job status and monitor resource usage in real time
  • Multiple workspaces: Maintain different computational sessions simultaneously
  • Mobile-friendly: Access resources from tablets and laptops with limited terminal capabilities
Callout

Perfect for Beginners

OnDemand is ideal if you’re new to HPC and prefer graphical interfaces. You can accomplish most common tasks without learning command-line tools. Point, click, and go!

The Interactive Apps dropdown menu open on the OnDemand dashboard. It lists a Desktop entry, then a GUIs group containing Autodock Vina, Comsol, Cryosparc, Gaussian, MATLAB, Mathematica, Qgis, Schrodinger, Stata and Tycho, then a Servers group containing Jupyter Notebook and RStudio Server.
Everything OnDemand can launch, in one menu.

OnDemand vs SSH Access

While SSH remains powerful for advanced users, OnDemand complements it by offering:

Feature OnDemand SSH
Learning curve Low Steep
File management Graphical Command-line
Interactive apps Direct support Manual setup
Job submission Web form Script writing
Device compatibility All browsers Terminal only

Accessing OnDemand

OnDemand at Pomona College is available at:

https://ondemand.hpc.pomona.edu/

Access requires: - Valid Pomona College credentials - DUO Multi-Factor Authentication (MFA) - Active HPC account approved by your PI

What You Can Do in OnDemand

File Management

  • Browse your home directory (/rhome/<myusername>)
  • Manage research data in /bigdata
  • Upload and download files
  • Create new directories
  • Edit text files

Job Submission

  • Submit batch jobs without writing SLURM scripts
  • Monitor job progress
  • View job output in real-time
  • Cancel running jobs

Interactive Applications

  • Launch Jupyter notebooks with GPU support
  • Start RStudio for statistical computing
  • Access web terminals for command-line work
  • Run custom interactive applications

System Resources

  • Check CPU and memory availability
  • Monitor cluster status
  • View node information
  • Track quota usage
Callout

Important: Authentication Required

You’ll need valid Pomona College credentials and DUO Multi-Factor Authentication to access OnDemand. Ensure your HPC account is active before attempting to log in. Contact if you have access issues.

Challenge

Challenge: Explore the Concept

Consider your typical workflow on Sagehen. Which tasks would benefit most from using OnDemand instead of SSH? Think about: 1. File management tasks 2. Data exploration work 3. Interactive development 4. Batch processing

There is no single correct answer, but common answers include: - File management: OnDemand is superior for uploading large datasets or browsing files - Data exploration: Jupyter notebooks through OnDemand are ideal for exploratory analysis - Learning: Beginners benefit from OnDemand’s graphical interface - Multiple tasks: OnDemand allows running several applications simultaneously in separate browser tabs - Mobile/remote: OnDemand works anywhere you have internet and a browser

Next Steps

Now that you understand what OnDemand offers, let’s explore the dashboard and learn to navigate its features.

Key Points
  • OnDemand is a web-based interface for accessing Sagehen HPC resources
  • It requires no SSH knowledge and works in any modern web browser
  • OnDemand is ideal for file management, interactive computing, and job monitoring
  • Pomona’s OnDemand is available at https://ondemand.hpc.pomona.edu/
  • OnDemand complements SSH access rather than replacing it

Content from Logging In and Dashboard Overview

Last updated on 2026-08-27 | Edit this page

Estimated time: 20 minutes

Overview

Questions

  • How do you log into OnDemand?
  • What does the dashboard look like?
  • How do you navigate between sections?
  • Where can you find session and account information?

Objectives

  • Log into OnDemand using Pomona credentials and DUO MFA
  • Identify the main areas of the dashboard
  • Navigate the top bar and side menu
  • Manage sessions and log out properly

Logging In

  1. Open your browser and go to https://ondemand.hpc.pomona.edu/
  2. Enter your Pomona username and password
  3. Complete DUO Multi-Factor Authentication at https://duo.pomona.edu
  4. The dashboard loads with your available resources

After authentication, your session persists for several hours. Your browser maintains the connection, allowing seamless navigation between sections.

The Pomona College sign-in page shown on a photograph of campus. A dialogue box asks for username@pomona.edu or username@mymail.pomona.edu, with a Next button, a link for people who cannot access their account, and a sign-in options link.
The Pomona College single sign-on page that OnDemand hands you off to.

Dashboard Layout

Top Navigation Bar

The top of every OnDemand page contains:

  • OnDemand Logo (left): Click to return to the dashboard from any page
  • Menu Hamburger (left): Opens/closes the main navigation sidebar
  • Help Links (right): Access documentation and support
  • Account Menu (far right): Your username with a dropdown for settings and logout
Callout

Tip: Bookmark the Dashboard

Save https://ondemand.hpc.pomona.edu/ as a bookmark for quick access. You will return to this page frequently during your work sessions.

Click the hamburger menu to reveal the main navigation:

  • Files – Home Directory, bigdata, scratch, tmpfs
  • Jobs – Active Jobs, Job Composer, Cluster
  • Accounts – Usage, Profile
  • Interactive Apps – Jupyter Notebook, RStudio, Web Terminal
  • Clusters – Sagehen HPC Shell Access
The Pomona College OnDemand dashboard. A top menu bar offers Apps, Files, Jobs, Clusters and Interactive Apps. The page below shows pinned apps grouped under three headings: Clusters, containing Pomona Cluster Shell Access and System Status; Files, containing Home Directory; and Interactive Apps.
The dashboard you land on: pinned apps grouped into Clusters, Files and Interactive Apps.

Status Indicators

OnDemand uses color-coded status indicators throughout the interface:

  • Green: Job is running, connection is active
  • Yellow/Orange: Job is queued or pending
  • Red: Error or job has failed
  • Gray: Completed or closed session

Session Management

  • View active sessions in the “My Sessions” section
  • Each session shows last access time and resource usage
  • Sessions automatically expire after inactivity
  • Check your Active Jobs regularly and close completed work to keep the dashboard clean

Logging Out

  1. Click your username (top right)
  2. Select “Sign Out”
  3. Always log out on shared or public computers

Contact if you have trouble logging in or accessing the dashboard.

Challenge

Challenge: Navigate the Dashboard

Log into OnDemand and complete these tasks: 1. Access the Files section and navigate to your home directory 2. Click on “Active Jobs” to view any running jobs 3. Open the “Cluster” view to check current node status 4. Check your disk quota in the Accounts/Usage section 5. Locate the button to start a Jupyter notebook

Can you find each of these sections? Write down what you observe about the current state of the cluster.

  1. Files section: Shows your files in /rhome/<myusername> with upload and download buttons
  2. Active Jobs: Displays a table with columns for job ID, name, user, status, time, and nodes
  3. Cluster view: Shows node status (idle, allocated, down) organized by partition
  4. Accounts/Usage: Displays CPU usage, storage quota, and job history
  5. Interactive Apps: A button or link to start Jupyter, RStudio, or Web Terminal

Common observations: - Different users may see different available partitions - Your quota depends on your research group allocation - The cluster status changes throughout the day as jobs complete and start

Key Points
  • Log in at https://ondemand.hpc.pomona.edu/ with Pomona credentials and DUO MFA
  • The top navigation bar has the logo, menu, help links, and account dropdown
  • The side menu organizes access through Files, Jobs, Apps, and Clusters
  • Color-coded indicators show job and resource status at a glance
  • Always log out on shared computers to protect your account

Content from Navigating Storage Locations

Last updated on 2026-08-27 | Edit this page

Estimated time: 15 minutes

Overview

Questions

  • What storage locations are available on Sagehen HPC?
  • How do /rhome and /bigdata differ from /scratch and /tmpfs?
  • What are the capacity limits for each storage location?
  • When should you use each storage location?

Objectives

  • Identify the four storage locations and their purposes
  • Understand which storage is persistent and which is temporary
  • Choose the right storage location for different types of data
  • Check storage quotas using quota_check.sh

Storage Locations on Sagehen HPC

Sagehen provides four storage locations, each designed for different use cases. Access them through the Files menu in the left navigation.

/rhome (Home Directory)

  • Path: /rhome/<myusername>
  • Purpose: Personal files, configuration, source code, small datasets
  • Persistent: Yes – files are preserved across sessions
  • Quota: Shared 1 TB lab quota with /bigdata (BeeGFS filesystem)
  • Access: Files > Home Directory

/bigdata (Shared Research Storage)

  • Path: /bigdata/lab/<labname>
  • Purpose: Large research datasets shared across lab members
  • Persistent: Yes – files are preserved across sessions
  • Quota: Shared 1 TB lab quota with /rhome (BeeGFS filesystem)
  • Access: Files > bigdata

/scratch (Fast Temporary Storage)

  • Path: /scratch
  • Speed: SSD (very fast)
  • Purpose: Intermediate computation results, temporary working files
  • Persistent: No – files are deleted when the job completes
  • Quota: Large, but temporary only
  • Access: Files > scratch

/tmpfs (In-Memory Storage)

  • Path: /tmpfs (per-node, not shared across nodes)
  • Speed: RAM-backed (fastest possible)
  • Purpose: Extremely fast I/O during active jobs
  • Persistent: No – files are deleted when the job completes
  • Capacity: Limited by node RAM
  • Access: Only available within running SLURM jobs
Callout

Choose Your Storage Wisely

Use /rhome for personal files and configuration. Store large research datasets in /bigdata for durability and sharing with collaborators. Use /scratch for temporary files needed during active jobs – but remember, scratch is non-persistent SSD and files are deleted when the job completes. Never rely on /scratch or /tmpfs for permanent storage!

Checking Your Quota

Because /rhome and /bigdata share a BeeGFS filesystem, the standard du command does not report usage correctly. Use the provided script instead:

BASH 

quota_check.sh

Run this from the Web Terminal or a shell session to see your current usage against the 1 TB lab quota.

Summary Table

Location Persistent Speed Shared Best For
/rhome Yes Standard Per-user Config, code, small files
/bigdata Yes Standard Per-lab Research data, results
/scratch No (job) SSD Yes Temp computation files
/tmpfs No (job) RAM Per-node High-speed I/O in jobs
Challenge

Challenge: Match the Storage Location

For each scenario below, choose the best storage location (/rhome, /bigdata, /scratch, or /tmpfs):

  1. You need to store a 200 GB genomics reference dataset that your whole lab uses.
  2. You are writing a Python script for your analysis pipeline.
  3. Your SLURM job produces 50 GB of intermediate files that are only needed during computation.
  4. You need the fastest possible I/O for reading millions of small temporary files during a single job.
  5. You want to keep your .bashrc and SSH configuration files.
  1. /bigdata – Large, shared dataset that needs to persist and be accessible to the whole lab.
  2. /rhome – Small personal file (source code) that should persist across sessions.
  3. /scratch – Intermediate files only needed during the job; deleted when the job completes.
  4. /tmpfs – RAM-backed storage provides the fastest I/O for temporary files within a single job.
  5. /rhome – Personal configuration files belong in your home directory.
Key Points
  • Sagehen has four storage locations: /rhome, /bigdata, /scratch, and /tmpfs
  • /rhome and /bigdata are persistent and share a 1 TB lab quota on BeeGFS
  • /scratch is non-persistent SSD storage deleted when the job completes
  • /tmpfs is RAM-backed storage deleted when the job completes
  • Use quota_check.sh to check usage – du does not work correctly on BeeGFS
  • Choose storage based on persistence needs, speed requirements, and sharing

Content from Uploading, Downloading, and Editing Files

Last updated on 2026-08-27 | Edit this page

Estimated time: 25 minutes

Overview

Questions

  • How do you upload files to Sagehen HPC through OnDemand?
  • How do you download files back to your local computer?
  • How do you create, rename, move, and delete files?
  • Can you edit files directly in the browser?

Objectives

  • Upload single and multiple files using the web interface
  • Download files and directories from Sagehen
  • Create directories and organize files
  • Edit text files directly in OnDemand
  • Manage file permissions through the browser

OnDemand File Manager

The file manager provides a graphical interface for managing files on Sagehen without using SCP or SFTP. Access it through the Files menu in the left navigation.

Uploading Files

Single File Upload

  1. Navigate to the desired directory in the file manager
  2. Click the Upload button (up arrow icon)
  3. Choose a file from your local computer
  4. Click Start Upload and wait for the progress bar to complete
The Pomona College OnDemand Files app showing the home directory. A toolbar across the top offers Open in Terminal, Refresh, New File, New Directory, Upload, Download, Copy Move, and Delete. The listing includes folders such as code, data, and projects plus several slurm output files.
The file manager in your home directory, with the Upload button in the toolbar.

Multiple File Upload

  1. Click the Upload button
  2. Select multiple files with Ctrl+Click (or Cmd+Click on Mac)
  3. All files upload simultaneously with individual progress indicators

Drag and Drop

  1. Open the file manager in your browser
  2. Drag files from your desktop or file explorer into the browser window
  3. Files upload automatically – faster than clicking buttons for batches

Tips: Upload large files to /bigdata, not /rhome (limited quota). Chrome and Firefox work best for uploads.

Downloading Files

  1. Open the file’s dropdown menu (⋮) and select Download, or click the file and use the toolbar Download button
  2. For multiple files, select them with checkboxes, then click Download Selected – they arrive as a ZIP archive
  3. Downloads go to your browser’s default Downloads folder

File Operations

Creating Directories

  1. Click New Directory
  2. Enter a name (e.g., “analysis_2026”)
  3. Click Create
A modal dialog titled New Directory over the OnDemand file manager, containing a single Directory name input field and two buttons, Close and OK.
The New Directory dialog — type a name and click OK.

Renaming

  1. Open the file’s dropdown menu (⋮) and select Rename
  2. Enter the new name and press Enter
A cropped view of the OnDemand file listing with one file selected and its dropdown menu open, showing five actions: View, Edit, Rename, Download, and a red Delete option.
Each file’s dropdown menu (⋮) offers View, Edit, Rename, Download, and Delete.

Moving Files

  1. Select the file(s), use the file’s dropdown menu (⋮), and choose Move – then navigate to the destination
  2. Or use the file’s dropdown menu (⋮) > Cut, navigate, then use the file’s dropdown menu (⋮) > Paste

Deleting

  1. Open the dropdown menu (⋮) for the file or folder and select Delete
  2. Confirm the deletion – verify you have backups before deleting important data

Inline Text Editing

OnDemand includes a built-in editor for text files (.txt, .sh, .py, .R, and similar):

  1. Select a text file and click Edit (or double-click)
  2. The editor opens in your browser with syntax highlighting
  3. Make changes and click Save

For files larger than 10 MB, use a command-line editor in the Web Terminal instead.

Managing Permissions

  1. Use the file’s dropdown menu (⋮) a file and select Properties or Permissions
  2. View current permissions (e.g., -rw-r--r--)
  3. Adjust read/write/execute for owner, group, and others

For executable scripts, ensure you mark them with execute permission.

Storage Quotas

Check your usage in Accounts > Usage or run quota_check.sh in the Web Terminal. Monitor regularly to avoid hitting limits mid-project.

Callout

Best Practice: Regular Cleanup

Check your storage quota monthly to avoid hitting limits mid-project. Remove old temporary files and archive completed work. If approaching quota limits, contact to request an increase rather than deleting valuable research data.

Challenge

Challenge: Upload, Organize, and Download

  1. Create a file on your local computer called “sample_data.txt” with some text
  2. Log into OnDemand and upload this file to /bigdata
  3. Create a new subdirectory called “project_2026”
  4. Move sample_data.txt into project_2026
  5. Download the file back to your computer to verify the transfer

Then check: How much storage are you currently using? Is your quota close to the 1 TB limit?

  1. Your sample_data.txt uploads to /bigdata (typically /bigdata/lab/<labname>)
  2. The new directory “project_2026” appears immediately
  3. Moving the file into the subdirectory works via drag or cut/paste
  4. Downloading returns the original file content intact
  5. Run quota_check.sh in the Web Terminal to see your current usage

Common observations: - Upload/download speeds depend on file size and your internet connection - Creating directories is instant - File operations in OnDemand are slower than command-line but more intuitive - Your /rhome and /bigdata share the same 1 TB lab quota

Key Points
  • Upload files via drag-and-drop, single select, or multi-select
  • Download single files directly or multiple files as a ZIP archive
  • Create, rename, move, and delete files through the graphical interface
  • Edit text files directly in the browser with syntax highlighting
  • File permissions can be viewed and changed through the web interface
  • Monitor your quota regularly with quota_check.sh or Accounts > Usage

Content from Shell Access and Command-Line Computing

Last updated on 2026-08-27 | Edit this page

Estimated time: 35 minutes

Overview

Questions

  • How do you access the command line through OnDemand?
  • What is a Web Terminal session?
  • How do you manage multiple terminal sessions?
  • What commands are available in OnDemand shell access?

Objectives

  • Launch and use the Web Terminal from OnDemand
  • Execute HPC commands without SSH
  • Manage multiple concurrent terminal sessions
  • Navigate the file system and manage resources using shell
  • Understand limitations and best practices for web-based terminals

Web Terminal Access

The Web Terminal in OnDemand provides a full command-line shell through your browser, eliminating the need to install SSH clients or learn command-line interfaces. It connects to a login node on Sagehen.

Callout

Important: Login vs. Compute Nodes

The Web Terminal connects to a login node, designed for small, quick tasks like editing files and submitting jobs. For intensive computations, submit SLURM jobs instead. Running computationally heavy tasks on the login node can affect other users.

Launching the Web Terminal

Method 1: Through Interactive Apps

  1. Open OnDemand at https://ondemand.hpc.pomona.edu/
  2. Navigate to “Interactive Apps” menu (left sidebar)
  3. Find and click “Web Terminal” or “Shell Access”
  4. Click “Launch”
  5. After a brief loading period, a terminal window opens in your browser

Method 2: Through Clusters Menu

  1. Go to Clusters menu
  2. Select “Sagehen HPC Shell Access”
  3. A terminal launches in a new tab

The terminal takes a few seconds to initialize the first time. Once loaded, you have a fully functional bash shell.

Using the Web Terminal

Basic Operations

The Web Terminal behaves like a standard Linux terminal. You can execute standard commands for:

  • Checking your current directory: pwd
  • Listing files: ls, ls -la
  • Changing directories: cd /rhome/<myusername>
  • Viewing file contents: cat, head
  • Creating directories: mkdir
  • Copying and moving files: cp, mv

Text Editing

You can use command-line editors from the web terminal:

  • Nano: A beginner-friendly editor (Ctrl+X to exit)
  • Vim: An advanced editor (:wq to save)
  • Default editor: edit command for text files

Job Submission

Submit SLURM jobs directly from the terminal with sbatch. View job status with squeue. Cancel jobs with scancel. Check cluster information with sinfo.

A typical job script includes:

#!/bin/bash
#SBATCH --job-name=MyJob
#SBATCH --partition=normal
#SBATCH --nodes=1
#SBATCH --time=01:00:00
#SBATCH --output=job.log

echo "Starting job at $(date)"
sleep 30
echo "Job complete"

Module Management

Load software modules for your work:

  • module avail: List available modules
  • module load: Load a specific module
  • module list: See currently loaded modules
  • module unload: Remove a module
  • module purge: Clear all modules

Multiple Terminal Sessions

Opening Multiple Terminals

You can open multiple terminal windows simultaneously through OnDemand. Each terminal is independent and can run different tasks. Use multiple terminals to monitor jobs while editing code, or to run multiple processes in parallel.

Browser tabs help organize your work:

  • Rename tabs to identify their purpose
  • Switch between tabs with mouse or keyboard shortcuts
  • Closing a tab closes that terminal session

File Operations in the Terminal

Working with Data

Standard Linux commands work for file operations:

  • wc -l: Count lines in files
  • grep: Search for patterns in text
  • sort and uniq: Organize data
  • tar and gzip: Compress and archive files
  • File manipulation: ls, cp, mv, rm, etc.

Creating Python Scripts

Python scripts can be created and executed from the terminal. Load Python with module load if needed, or use the system Python. Create files with editors or echo commands, then execute with python3.

Running Shell Scripts

Shell scripts can automate repetitive tasks. Create with a text editor, make executable with chmod +x, then run with ./scriptname.

Monitoring and Resource Management

Check System Resources

  • top: View current CPU and memory usage
  • sinfo: See available nodes and partitions
  • squeue: View all jobs (or -u $USER for your jobs)
  • scontrol show job: Get detailed job information
  • hostname: See which machine you’re on

Environment Variables

Access important information about your environment and current job:

  • $USER: Your username
  • $HOME: Your home directory path
  • $SLURM_NODELIST: Nodes allocated to your job
  • hostname: The current machine name

Terminal Limitations and Considerations

The Web Terminal has some important limitations:

  • Interactive programs may have limited support
  • Can only connect to systems accessible from Sagehen
  • Long-idle sessions may disconnect
  • Copy-paste works with Ctrl+C/Ctrl+V
  • Personal SSH keys are available for other systems

Best Practices for Web Terminal

  1. Don’t leave idle terminals open; close ones you’re not actively using
  2. Use screen or tmux for long-running processes that survive disconnections
  3. Save work frequently in case of unexpected disconnections
  4. Write temporary files to /scratch, not /rhome
  5. Monitor resources with squeue before launching new jobs
  6. Cancel idle jobs with scancel to free resources
Callout

Pro Tip: Use screen or tmux

For long-running processes, use screen or tmux to create persistent sessions that survive browser disconnections. Start your process inside a screen session, then detach with Ctrl+A+D. Reconnect later even after closing your terminal.

Troubleshooting Terminal Issues

  • Slow performance: Try a different browser or check network
  • Copy-paste issues: Try Ctrl+Shift+C/Ctrl+Shift+V
  • Disconnections: Open new terminal and check squeue for jobs
  • Command not found: Load required module with module load
  • Permission denied: Check file permissions with ls -l
Challenge

Challenge: Command-Line Computing

Use the Web Terminal to complete these tasks:

  1. Navigate to your /rhome directory and list all files
  2. Create a new directory called “my_analysis”
  3. Create a simple Python script that prints “Hello from Sagehen!”
  4. Run the Python script from the terminal
  5. Check the current status of the Sagehen HPC cluster with sinfo
  6. View any jobs you have running with squeue -u $USER

Record what you observe, especially:

  • How many nodes are in the cluster?
  • What partitions are available?
  • Are you currently running any jobs?

After completing this challenge:

  1. Your /rhome directory contains configuration files and personal data
  2. The new directory appears in listings with mkdir
  3. Your Python script executes successfully, printing the output
  4. sinfo shows the cluster composition with nodes in various partitions
  5. squeue -u $USER shows your jobs if running, or an empty table
  6. You see partition information and node status

Common observations:

  • Most nodes are idle or allocated during normal times
  • Jobs have IDs starting from recent numbers
  • Python/Python3 is available without module loading
  • The terminal responds quickly to most commands
  • Complex file operations work just like SSH access
  • Cluster status changes throughout the day as jobs complete

Accessing Sagehen HPC via SSH (Alternative)

While the Web Terminal is convenient, some users prefer SSH for direct access:

ssh <myusername>@sagehen.hpc.pomona.edu
ssh -i ~/.ssh/id_rsa <myusername>@sagehen.hpc.pomona.edu

SSH provides the same shell access but requires pre-configured authentication on your local computer.

Key Points
  • Web Terminal provides full command-line access through your browser
  • Launch from Interactive Apps or Clusters menu in OnDemand
  • Run standard Linux commands, submit SLURM jobs, and manage files
  • Multiple terminal sessions can run independently for parallel work
  • Use screen/tmux for long-running processes that survive disconnections
  • Web Terminal is convenient for beginners; SSH is available for advanced users

Content from Launching Jupyter Notebooks

Last updated on 2026-08-27 | Edit this page

Estimated time: 25 minutes

Overview

Questions

  • How do you launch a Jupyter notebook from OnDemand?
  • How do you configure resources for your notebook session?
  • How do you use notebook cells, run code, and write documentation?
  • How do you install additional Python packages?

Objectives

  • Launch a Jupyter notebook session from OnDemand
  • Configure CPU, memory, and time for your session
  • Create and execute code and markdown cells
  • Upload data and install packages in the notebook environment

Launching Jupyter

  1. Click Interactive Apps in the left sidebar
  2. Select Jupyter Notebook (or Jupyter Lab)
  3. Configure your resources:
    • CPU cores: 1–4 for most work
    • Memory: 4–16 GB depending on dataset size
    • Time: 1–8 hours
    • GPU (optional): Select if you need accelerated computing
  4. Click Launch
  5. Wait for allocation (typically 1–5 minutes)
  6. Click Connect to Jupyter when the session is ready
The Jupyter Notebook launch form in OnDemand. Fields include Pom Partitions set to amd, with notes that amd is the default partition allowing up to 720 hours per job, gpu is the GPU partition also allowing 720 hours, and short is limited to 2 hours but starts with little or no wait. Below are number of hours, Jupyter version 4.4.0, Jupyter type lab, number of CPU cores, memory in gigabytes, an email checkbox, and a Launch button.
The launch form is where you choose a partition and ask for resources – the same decisions as an sbatch script, as a web form.
Callout

First Launch Takes Time

When you request Jupyter, expect to wait 1–5 minutes for resource allocation, especially during busy hours. The system is requesting compute nodes for your session. Subsequent launches typically start faster.

Using Jupyter Notebooks

Creating a Notebook

Click New then Python 3 to create a new notebook. Each notebook is a .ipynb file saved in your working directory.

Cell Types and Execution

  • Code cells: Write Python code and press Shift+Enter to execute
  • Markdown cells: Write formatted text, headings, and documentation
  • Output: Results, tables, and plots appear directly below each cell

A common first cell:

PYTHON 

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
print("Libraries loaded successfully!")

Uploading Data

  1. Use the Jupyter file browser on the left to navigate to your data
  2. Or click the Upload button within Jupyter
  3. Read files in your notebook:

PYTHON 

data = pd.read_csv('/bigdata/lab/<labname>/data.csv')
print(data.head())

Use absolute paths (like /bigdata/lab/<labname>/) to avoid confusion about your working directory.

Installing Packages

Install additional packages directly from a notebook cell:

PYTHON 

!pip install scikit-learn
!pip install seaborn

Or open a terminal in Jupyter (New > Terminal) for more complex installations.

Session Management

  • Sessions expire after idle time or when the requested time limit is reached
  • Save your work frequently with Ctrl+S
  • Download important notebooks before your session expires
  • Check remaining time in the OnDemand Active Jobs view

Use batch jobs (sbatch) for computations longer than 8 hours instead of interactive Jupyter sessions.

Challenge

Challenge: Create Your First Notebook

  1. Launch a Jupyter notebook from OnDemand with 1 core, 4 GB memory, and 1 hour

  2. Create a new Python 3 notebook

  3. In cell 1, import numpy, pandas, and matplotlib

  4. In cell 2, create a sample dataset:

    PYTHON 

    data = np.random.normal(loc=50, scale=10, size=500)
df = pd.DataFrame({'values': data})
print(df.describe())

  5. In cell 3, create a histogram:

    PYTHON 

    plt.hist(df['values'], bins=25, edgecolor='black')
plt.xlabel('Value')
plt.ylabel('Frequency')
plt.title('Sample Distribution')
plt.show()

  6. Save your notebook

  1. Jupyter launches after 1–5 minutes of resource allocation
  2. A new notebook opens with an empty code cell
  3. All three libraries import without errors
  4. df.describe() prints count, mean, std, min, max, and quartiles
  5. A histogram appears inline showing a normal distribution centered around 50
  6. The notebook saves as an .ipynb file in your working directory

If your session takes a long time to start, the cluster may be busy – try reducing your resource request or launching during off-peak hours.

Key Points
  • Launch Jupyter from Interactive Apps with configurable CPU, memory, and time
  • Expect 1–5 minutes for initial resource allocation
  • Use code cells for Python and markdown cells for documentation
  • Install packages with !pip install directly in notebook cells
  • Save work frequently and download notebooks before sessions expire
  • Use batch jobs for computations longer than 8 hours

Content from RStudio and GPU Resources

Last updated on 2026-08-24 | Edit this page

Estimated time: 25 minutes

Overview

Questions

  • How do you launch RStudio from OnDemand?
  • What is the RStudio interface layout?
  • How do you request GPU resources for interactive apps?
  • How do you verify GPU access in Jupyter or R?

Objectives

  • Launch an RStudio session and navigate the interface
  • Execute R code and install R packages
  • Request GPU resources when launching interactive apps
  • Verify GPU availability and use GPU-accelerated libraries

Launching RStudio

  1. Click Interactive Apps in the left sidebar
  2. Select RStudio Server
  3. Configure resources (similar to Jupyter – cores, memory, time)
  4. Click Launch and wait for allocation
  5. Click Connect to RStudio when the session is ready

RStudio Interface Layout

Once launched, RStudio provides four panels:

  • Script editor (top-left): Write and save R scripts
  • Console (bottom-left): Execute R commands interactively
  • Environment/History (top-right): View loaded variables and command history
  • Files/Plots/Packages (bottom-right): Browse files, view plots, manage packages

Running R Code

Write code in the script editor or directly in the console:

First create a small sample file to work with, then read it back, summarize, and plot (in real work you would read your own data from /bigdata/lab/<labname>/ instead):

R 

library(tidyverse)

# Create a small sample dataset in your home directory
dir.create("~/rstudio-demo", showWarnings = FALSE)
write.csv(data.frame(variable1 = rnorm(8), variable2 = rnorm(8)),
          "~/rstudio-demo/data.csv", row.names = FALSE)

df <- read.csv("~/rstudio-demo/data.csv")
summary(df)

ggplot(df, aes(x = variable1, y = variable2)) +
  geom_point() +
  theme_minimal()
The four-panel RStudio Server interface. The editor and console show R code writing an eight-row sample CSV to the home directory, loading tidyverse, reading the file back, and printing a summary. The Plots pane displays a scatter plot and the Environment pane lists a data frame of 8 observations of 2 variables.
A working version of the example: write a small sample CSV first, then read, summarize, and plot it — all four RStudio panes in action.

Installing R Packages

In the console:

R 

install.packages("ggplot2")
install.packages("dplyr")
library(ggplot2)

Or use the Packages tab in the bottom-right panel to install graphically.

Callout

Save Your Work

Interactive sessions have time limits. Save scripts and results frequently with Ctrl+S. Download important files or save to /bigdata for durability. Do not wait until your session expires – save early and often.

Requesting GPU Resources

When launching Jupyter or RStudio, select GPU options in the resource form:

  • GPU type: A100 (80 GB) or L40S (48 GB)
  • GPU count: Usually 1 for interactive work
  • Partition: Select gpu to access GPU nodes

Verifying GPU in Jupyter (Python)

PYTHON 

import torch
print(torch.cuda.is_available())       # Should print True
print(torch.cuda.get_device_name(0))   # Shows GPU model

import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))

Verifying GPU in R

R 

library(tensorflow)
tf$config$list_physical_devices('GPU')

GPU-Accelerated Libraries

Once GPU is verified, use it for computation:

PYTHON 

import torch
x = torch.randn(1000, 1000).cuda()
y = torch.mm(x, x)

Monitor GPU usage from a terminal with nvidia-smi.

Resource Guidelines

  • Small datasets (< 100 MB): 1–2 cores, 4 GB memory, no GPU
  • Medium datasets (100 MB – 10 GB): 2–4 cores, 8–16 GB memory
  • GPU work: Request only when needed – GPUs are a shared resource
  • Close sessions when finished to free resources for others
Challenge

Challenge: Launch RStudio and Explore

  1. Launch RStudio from OnDemand with 1 core, 4 GB memory, and 1 hour

  2. In the console, load the tidyverse package: library(tidyverse)

  3. Create a sample data frame:

    R 

    df <- data.frame(
  x = rnorm(100),
  y = rnorm(100),
  group = sample(c("A", "B"), 100, replace = TRUE)
)

  4. Create a scatter plot:

    R 

    ggplot(df, aes(x = x, y = y, color = group)) +
  geom_point() +
  theme_minimal() +
  labs(title = "Sample Scatter Plot")

  5. Save the plot with ggsave("scatter.png")

RStudio Server showing the challenge solution. The console echoes code creating a 100-row data frame with x, y, and a two-level group column. The Plots pane displays a chart titled Sample Scatter Plot with points in two colors for groups A and B, and the Environment pane confirms 100 observations of 3 variables.
The expected challenge result: 100 random points colored by group, with the data frame in the Environment pane.
  1. RStudio launches after resource allocation (1–5 minutes)
  2. tidyverse loads with some startup messages (this is normal)
  3. The data frame appears in the Environment panel showing 100 observations of 3 variables
  4. A scatter plot appears in the Plots panel with points colored by group
  5. The file “scatter.png” is saved in your working directory and visible in the Files panel

If tidyverse is not installed, run install.packages("tidyverse") first – this may take a few minutes.

Key Points
  • Launch RStudio from Interactive Apps with configurable resources
  • RStudio provides script editor, console, environment, and files/plots panels
  • Install R packages from the console or the Packages tab
  • Request GPU resources by selecting the gpu partition and GPU type (A100, L40S, RTX PRO 6000)
  • Verify GPU access with torch.cuda.is_available() in Python or tf in R
  • Save your work frequently – sessions have time limits

Content from Submitting Batch Jobs

Last updated on 2026-08-27 | Edit this page

Estimated time: 25 minutes

Overview

Questions

  • How do you submit batch jobs through OnDemand?
  • What is the difference between Quick Submit and uploading a script?
  • How do you write a basic SLURM job script?
  • How do you submit GPU jobs and job arrays?

Objectives

  • Access the Job Composer in OnDemand
  • Submit jobs using Quick Submit and script upload
  • Write a basic SLURM batch script
  • Submit GPU jobs and job arrays

Accessing the Job Composer

  1. Click Jobs in the left sidebar
  2. Select Job Composer
  3. You see any previously created jobs
  4. Click New Job to create and submit a new job
Callout

Test Your Scripts First

Before submitting a job to compute nodes, test your script logic in the Web Terminal or with a small test dataset. This saves compute resources and helps you catch errors before they run on expensive hardware.

Quick Submit

  1. Click New Job
  2. Fill in the job details:
    • Job name: A descriptive name (e.g., “Analysis_2026”)
    • Partition: amd, gpu, or short (amd and gpu allow up to 30 days / 720 hours; short has a shorter walltime — check sinfo -p short)
    • Nodes: Usually 1
    • Time limit: Hours and minutes needed
    • Memory: Total RAM (leave blank for default)
  3. Paste your script content in the editor
  4. Click Submit

Upload a Script

  1. Click New Job then Upload
  2. Select a SLURM script file from your computer
  3. Review and modify parameters if needed
  4. Click Submit

Writing a SLURM Script

A basic batch script:

BASH 

#!/bin/bash
#SBATCH --job-name=MyJob
#SBATCH --partition=amd
#SBATCH --nodes=1
#SBATCH --ntasks=4
#SBATCH --time=02:00:00
#SBATCH --output=job.log
#SBATCH --error=job.err

cd $SLURM_SUBMIT_DIR
module load miniconda3
python3 analysis.py

Key directives: --job-name identifies the job, --partition selects the queue, --time sets the wall-clock limit, and --output/--error capture stdout and stderr.

GPU Job Submission

To request GPU resources, use the gpu partition with --gres:

BASH 

#!/bin/bash
#SBATCH --job-name=GPUJob
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --gres=gpu:1
#SBATCH --time=04:00:00
#SBATCH --output=gpu_job.log

module load miniconda3 cuda/12.2.1
conda activate pytorch_env   # your conda env with PyTorch (see the ML workshop)
python3 train_model.py

Request a specific GPU type with --gres=gpu:a100:1, --gres=gpu:l40s:1, or --gres=gpu:rtxpro6000:1.

Job Arrays

Submit multiple similar jobs at once:

BASH 

#!/bin/bash
#SBATCH --job-name=analysis
#SBATCH --array=1-10
#SBATCH --partition=amd
#SBATCH --time=01:00:00

python3 analysis.py input_${SLURM_ARRAY_TASK_ID}.txt

Each array task gets a unique $SLURM_ARRAY_TASK_ID (1 through 10 in this example), letting you process multiple input files with one submission.

Challenge

Challenge: Submit and Monitor a Job

  1. Use Job Composer to create a job that:
    • Prints “Job starting at $(date)”
    • Sleeps for 30 seconds
    • Prints “Job completed at $(date)”
  2. Submit the job to the amd partition
  3. Watch the job progress in Active Jobs
  4. When complete, view the output log
  5. Note how long the job took (queue wait + runtime)
  1. Your script runs successfully on the compute node
  2. Active Jobs shows the job progressing: Pending then Running then Completed
  3. The output log shows your date stamps and confirms completion
  4. Total time includes queue wait (seconds to minutes) plus the 30-second sleep
  5. Job ID is a unique number assigned by SLURM

Common observations:

  • Queue time varies based on cluster load
  • Output appears in the log file after job completion
  • The amd and gpu partitions each allow jobs up to 30 days (720 hours); the short partition has a shorter max walltime — check sinfo -p short for the current limit
Key Points
  • Access the Job Composer through Jobs in the left sidebar
  • Quick Submit lets you paste a script; Upload lets you use an existing file
  • SLURM directives control partition, time, nodes, and output locations
  • Use –partition=gpu and –gres=gpu:1 for GPU jobs
  • Job arrays let you submit many similar jobs with one script
  • Sagehen HPC offers three partitions: amd, gpu (both with 30-day max walltime), and short (shorter max walltime, good for quick test/debug jobs — check sinfo -p short)

Content from Monitoring Jobs and Troubleshooting

Last updated on 2026-08-27 | Edit this page

Estimated time: 25 minutes

Overview

Questions

  • How do you monitor running jobs in OnDemand?
  • Where do you find job output and error logs?
  • How do you debug a failed job?
  • What are typical queue wait times and job limits?

Objectives

  • Use the Active Jobs view to monitor job status
  • Retrieve and interpret output and error logs
  • Debug common job failures using error messages
  • Understand job limits, queue status, and scheduling basics

Active Jobs View

Navigate to Jobs then Active Jobs to see all your jobs. Each entry shows:

  • Job ID: Unique identifier assigned by SLURM
  • Name: Job name from your script
  • Status: Running (green), Pending (yellow), Completed (blue), Failed (red)
  • Time: How long the job has been running
  • Nodes: Which compute node(s) the job is using

Job Details

Click on any job to see:

  • Partition and node assignments
  • Resource allocation (CPU cores, memory, GPUs)
  • Time information (submission, start, estimated end)
  • Links to output and error files

Real-Time Monitoring

While a job is running:

  1. Click on the job in Active Jobs
  2. Click View Output or View Error
  3. Refresh your browser to see the latest output
  4. Useful for tracking progress and catching problems early

Retrieving Output and Error Logs

After a job completes, OnDemand saves:

  • job.log (or your --output filename): Standard output from your script
  • job.err (or your --error filename): Error messages and warnings
  • Result files: Any files your script created

Access these through the Active Jobs detail view or the Files section.

Debugging Failed Jobs

When a job fails, check the error log for these common messages:

Error Message Cause Fix
“module not found” Module load failed Check module name with module avail
“file not found” Wrong input path Use absolute paths like /bigdata/lab/<labname>/file
“out of memory” Job needs more RAM Increase --mem and resubmit
“timeout” / “time limit” Job exceeded wall time Increase --time (max 720 hours)
“permission denied” Wrong file permissions Check with ls -l and fix with chmod

Fix the issue in your script, then resubmit through Job Composer.

Canceling Jobs

  1. Find the job in Active Jobs
  2. Click the red delete (trash-can) button in the job’s row
  3. Confirm — a green “Successfully deleted ” banner appears
  4. The job stops immediately and resources are freed

You can also cancel from the Web Terminal with scancel <job_id>.

The Pomona College OnDemand Active Jobs page. A green banner reads Successfully deleted 38395. The table lists a completed Jupyter session on the amd queue and a running RStudio Server session on the short queue, and the mouse hovers over a red trash-can button used to cancel the running job.
The Active Jobs view right after canceling a session — the trash-can button in the Actions column stops a running job. Note the status colors: Completed is green, Running is blue.

Job Constraints and Limits

Sagehen HPC enforces limits for fair resource sharing:

  • Time: Max 720 hours (30 days) per job on amd and gpu partitions; the short partition has a shorter max walltime — check sinfo -p short
  • GPU limit: GPU usage limits per user are configured in SLURM; check sacctmgr show user $USER or contact for current limits.
  • Queue position: Based on submission time and fairness policies

Queue Status and Scheduling

Check cluster status through Jobs then Cluster, or run sinfo in the Web Terminal. The view shows available nodes by partition, node status (idle, allocated, down), and current queue depth.

Callout

Check Cluster Status

Before submitting large jobs, run sinfo in the Web Terminal or check Jobs > Cluster to see available resources. Submit during off-peak hours (evenings and weekends) for faster execution. Small jobs typically start faster than large ones.

Typical Wait Times

  • amd partition: Usually minutes to hours depending on cluster load
  • gpu partition: May wait longer when GPUs are in high demand
  • short partition: Typically the fastest to start because the scheduler can back-fill short jobs between longer ones
  • Off-peak hours: Jobs typically start faster (evenings and weekends)
  • Smaller resource requests: Generally queue for less time

Scheduling Basics

  1. Your job enters the queue when submitted
  2. SLURM allocates resources based on job requirements, availability, and fairness
  3. When resources become available, the job starts
  4. The job completes and resources are freed for others

Priority depends on your recent usage history, number of running jobs, and time since your last job completed.

Challenge

Challenge: Debug a Failed Job

A job submitted the following script but failed. Review the error log below and identify the problem:

#!/bin/bash
#SBATCH --job-name=analysis
#SBATCH --partition=amd
#SBATCH --time=00:10:00
#SBATCH --output=analysis.log
#SBATCH --error=analysis.err

module load miniconda3
cd /bigdata/lab/<labname>/project
python3 run_analysis.py --input data.csv --output results.csv

Error log (analysis.err):

/var/spool/slurmd/job12345/slurm_script: line 9:
cd: /bigdata/lab/<labname>/project: No such file or directory
python3: can't open file '/bigdata/lab/<labname>/project/run_analysis.py':
[Errno 2] No such file or directory

What went wrong, and how would you fix it?

The error shows two problems caused by the same root issue:

  1. The directory /bigdata/lab/<labname>/project does not exist. Either the lab name is wrong or the directory was never created.
  2. The Python script cannot be found because the cd failed and the script is not at the default path.

To fix: - Verify the correct lab directory name (check in OnDemand Files > bigdata) - Create the directory if it does not exist: mkdir -p /bigdata/lab/<labname>/project - Confirm the script run_analysis.py and data.csv are in that directory - Resubmit the job after fixing the path

Always use ls to verify paths before submitting jobs.

Key Points
  • Active Jobs shows real-time status with color-coded indicators
  • Click any job to see details, output logs, and error logs
  • Common failures include wrong paths, missing modules, and insufficient resources
  • Cancel jobs from Active Jobs or with scancel in the terminal
  • Sagehen offers three partitions: amd and gpu (each up to 720 hours / 30 days) and short (shorter max walltime, good for quick test/debug jobs — check sinfo -p short)
  • Check cluster status before submitting and prefer off-peak hours for faster starts