Handling PUBLIC Data

Estimated time: 25 minutes

Overview

Questions

• What access controls are needed for Public data?
• Where should Public data be stored on Sagehen?
• What permissions should I set for Public files?

Objectives

• Understand the specific handling requirements for PUBLIC data
• Implement access controls appropriate to Public data
• Set up storage for Public data on Sagehen HPC

PUBLIC Data Handling Requirements

---

Access Control

Who can access: Anyone (unrestricted) or lab group (shared)
Default file permissions: world-readable (755) or group-readable (750)

No special access control is needed. Public data is by definition available to the world or freely shared within groups.

Storage

• Location: Anywhere on Sagehen (/rhome or /bigdata)
• Encryption: Not required (not recommended; unnecessary overhead)
• File system: Standard Sagehen filesystem

Example: Published Dataset

BASH

# Published paper's supplementary data
/bigdata/lab/<labname>/published_paper_2025/
  ├── README.md (DOI, citation info)
  ├── data_clean.csv
  ├── analysis.R
  └── figures/

# Permissions (world-readable)
ls -l /bigdata/lab/<labname>/published_paper_2025/
-rw-r--r-- 1 pi_user lab_group  500M data_clean.csv
-rw-r--r-- 1 pi_user lab_group   50K analysis.R

Audit Logging

Not required for Public data.

Retention

No special requirements. Retain as long as useful for reproducibility (typically as long as the paper is cited, indefinitely for reproducibility).

Storage Layout for Public Data

---

Fully Public Data

BASH

/bigdata/lab/<labname>/public_datasets/
├── published_paper_data/
│   ├── data_clean.csv
│   └── analysis.R
└── course_materials/
    ├── lecture_notes.pdf
    └── sample_data.csv

# Permissions (world-readable)
chmod 644 /bigdata/lab/<labname>/public_datasets/*

Public Data Shared within Group

BASH

/bigdata/lab/<labname>/projects/
├── exp_2025_01/
│   ├── raw_data.h5
│   └── analysis/
└── exp_2025_02/
    ├── images/
    └── processed/

# Permissions (group-readable, 750)
chmod 750 /bigdata/lab/<labname>/projects/
find /bigdata/lab/<labname>/projects/ -type f -exec chmod 640 {} \;

Practical Exercise: Set Up Public Data Access

---

BASH

# Check your lab group
groups

# Set directory permissions for Public shared data
chmod g+rx /bigdata/lab/<labname>/public/
find /bigdata/lab/<labname>/public/ -type f -exec chmod 640 {} \;
find /bigdata/lab/<labname>/public/ -type d -exec chmod 750 {} \;

# Verify
ls -ld /bigdata/lab/<labname>/public/
ls -l /bigdata/lab/<labname>/public/data.csv

Challenge

Challenge 7.1: Public Data Setup

You have two datasets to store: 1. Published paper data (CSV, 5 GB) – fully public 2. Lab meeting notes (Markdown, 5 MB) – shared with lab group

For each: - Choose the storage location (/rhome or /bigdata) - Set the correct permissions - Explain whether encryption is needed

Show me the solution

1. Published data → /bigdata (large dataset) → permissions 755 → no encryption needed
2. Lab meeting notes → /bigdata (shared with group) → permissions 750 → no encryption needed

Key Points

• Public data: World-readable (755) or group-readable (750); no encryption needed
• Use /bigdata for large public datasets; /rhome is acceptable for small files
• No audit logging or special retention required for Public data
• Still verify data was approved for public release before sharing