Key Points

The AI Landscape for Researchers

  • HPC is essential for sensitive data, large models, and reproducible research
  • Cloud AI is easy but sends data externally; local AI keeps data private
  • Model size, memory requirements, and accuracy vary widely across AI tools
  • Data classification – not model capability – should drive tool selection

Types of AI Models

  • Useful local LLMs range from about 4B to over 100B parameters
  • Choose on licence, size and whether the download is gated – not on leaderboards
  • Quantization (4-bit) reduces memory by 75% with minimal quality loss
  • Vision models (YOLO, Stable Diffusion) and scientific ML have specialized uses
  • Start with the smallest model that meets your quality threshold and scale up

Pomona's AI Use Policy

  • Pomona policy requires data classification before AI tool selection
  • PUBLIC data may use external AI; PROPRIETARY requires local AI or written permission; RESTRICTED data must never go to external AI
  • Violations can result in warnings, retractions, or legal consequences
  • When unsure, contact before proceeding

Data Classification for AI

  • PUBLIC data: safe with external AI tools
  • PROPRIETARY data: local AI only, or external with explicit written permission
  • RESTRICTED data (FERPA, medical, financial, export-controlled): never external AI
  • Use the decision tree to classify before selecting any AI tool
  • Redact personal identifiers before sharing data with any external tool

Sensitive Data and AI

  • FERPA absolutely prohibits sending student data to external AI
  • HIPAA requires de-identification and local AI for medical data
  • Export-controlled research must stay on US infrastructure (Sagehen)
  • Use gocryptfs for encryption of restricted data at rest
  • Document all AI usage with a pre-research governance checklist
  • Verify all AI-generated outputs before using in publications

Local vs Cloud AI

  • Cloud AI is faster and higher quality but sends data externally
  • Local AI on Sagehen keeps data private and is cost-effective at scale
  • Data classification determines which approach is allowed
  • Cloud AI is never acceptable for RESTRICTED or uncleared PROPRIETARY data
  • Build workflows that use cloud AI for public data and local AI for sensitive data

Running Models on Sagehen HPC

  • Load the anaconda3 and cuda modules before GPU work
  • Use conda environments for reproducible, isolated Python setups
  • Configure HF_HOME to /bigdata to avoid filling home quota with models
  • Use srun for interactive GPU testing and sbatch for batch jobs
  • Start with smaller GPUs and scale up once your workflow is validated

Running Local LLMs on Sagehen HPC

  • Open-weight LLMs (gpt-oss, Qwen, Mistral, Phi) run locally on Sagehen GPUs
  • Prefer ungated, permissively licensed models – no access request to wait on
  • 4-bit quantization reduces memory by 75% with minimal quality loss
  • Set HF_HOME to lab storage so weights do not fill your /rhome quota
  • Build private pipelines for restricted data that never leaves Sagehen
  • Log AI usage for Pomona compliance with timestamps, model, revision and data type

Responsible AI Practices

  • Always disclose AI usage specifically in methods sections and code comments
  • Verify all AI-generated claims, citations, and code before use
  • AI models hallucinate regardless of whether they are cloud or local
  • Transparent disclosure builds credibility; concealment destroys it
  • Consider environmental impact when choosing model size and scale

Ethical Considerations and Next Steps

  • AI systems encode biases from training data, labeling, and measurement choices
  • Fairness audits should test accuracy across demographic groups before deployment
  • Write broader impact statements acknowledging risks and mitigations
  • Maintain research integrity: AI accelerates work but does not replace verification
  • Start your AI journey with data classification, small models, and thorough documentation