Python Fundamentals in Notebooks

Estimated time: 30 minutes

Overview

Questions

• How do I write Python code in notebooks?
• How do I import and use common data science libraries?
• How do I load data from CSV and other formats?
• What’s special about IPython magic commands?

Objectives

• Write and execute Python code in notebook cells
• Import and use NumPy, pandas, and matplotlib
• Load data from CSV and other file formats
• Use IPython magic commands for enhanced functionality

Python in Jupyter

---

Jupyter notebooks run Python code almost identically to regular Python scripts, but with some enhancements. Let’s explore the main differences and capabilities.

Basic Python syntax

Standard Python works as expected:

PYTHON

# Variables and types
name = "Alice"
age = 30
scores = [85, 92, 78, 95]
data_dict = {"name": "Bob", "age": 25}

# Loops and conditionals
for score in scores:
    if score >= 90:
        print(f"{score} is excellent!")

# Functions
def calculate_average(values):
    return sum(values) / len(values)

avg = calculate_average(scores)
print(f"Average: {avg:.2f}")

This all works in Jupyter exactly as it would in a script.

Importing Libraries

---

Most data science work relies on libraries. Common ones:

NumPy (numerical computing)

PYTHON

import numpy as np

# Create arrays
arr = np.array([1, 2, 3, 4, 5])
matrix = np.ones((3, 4))

# Array operations
result = arr * 2
print(result)  # [2 4 6 8 10]

# Statistics
mean = np.mean(arr)
std = np.std(arr)

pandas (data manipulation)

PYTHON

import pandas as pd

# Create a DataFrame (like an Excel table)
data = {
    'name': ['Alice', 'Bob', 'Carol'],
    'age': [30, 25, 28],
    'salary': [50000, 45000, 55000]
}
df = pd.DataFrame(data)

# Access columns
print(df['name'])

# Statistics
print(df.describe())

# Filter rows
high_earners = df[df['salary'] > 50000]

matplotlib (visualization)

PYTHON

import matplotlib.pyplot as plt

# Create a line plot
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 5, 8]
plt.plot(x, y)
plt.xlabel('X axis')
plt.ylabel('Y axis')
plt.title('Simple Plot')
plt.show()  # Important! Displays the plot

Loading Data from Files

---

A typical workflow: Load data from CSV (or other formats) and explore it.

From CSV files

PYTHON

import pandas as pd

# Load CSV file (use full path for clarity)
df = pd.read_csv('/rhome/<myusername>/data/experiments.csv')

# See first few rows
print(df.head())

# Get information about the DataFrame
print(df.info())

# Summary statistics
print(df.describe())

# Check shape (rows, columns)
print(f"Data has {df.shape[0]} rows and {df.shape[1]} columns")

From other formats

pandas supports many formats:

PYTHON

# Excel file
df = pd.read_excel('data.xlsx')

# JSON
df = pd.read_json('data.json')

# HDF5 (efficient for large datasets)
df = pd.read_hdf('data.h5', key='dataset')

# Pickle (Python-specific, fast)
df = pd.read_pickle('data.pkl')

IPython Magic Commands

---

Jupyter uses IPython (Interactive Python), which includes special commands called “magic” commands. These start with % (line magic) or %% (cell magic).

Callout

Commonly used magic commands

Timing code:

PYTHON

%timeit sum(range(1000000))    # Time a single line
%time x = sum(range(1000000))  # Time once (not averaged)

Displaying output:

PYTHON

%matplotlib inline             # Show plots inline (usually default)
%whos                          # List all variables in memory

Running external files:

PYTHON

%run my_script.py              # Run a Python script
%load my_code.py               # Load code from file into a cell

Output Handling

---

By default, Jupyter only displays the last output in a cell. To display multiple outputs:

PYTHON

# Method 1: Use print()
print("First output")
print("Second output")

# Method 2: Use display()
from IPython.display import display
display("First output")
display("Second output")

# Method 3: Semicolon suppresses output
x = 5
y = 10
z = x + y;  # Semicolon suppresses output

Challenge

Challenge 1: Load and explore data

1. Create a new notebook cell and load some sample data:

   PYTHON

   import pandas as pd
   import numpy as np
   
   # Create sample dataset
   np.random.seed(42)
   df = pd.DataFrame({
       'date': pd.date_range('2024-01-01', periods=100),
       'temperature': np.random.normal(65, 10, 100),
       'humidity': np.random.normal(60, 15, 100),
       'wind_speed': np.random.exponential(5, 100)
   })

2. In subsequent cells, explore the data:

   • Print df.head() to see first rows
   • Print df.info() to see data types
   • Print df.describe() for statistics
   • Calculate mean temperature and humidity

Document each step with markdown cells explaining what you’re doing.

Show me the solution

Cell 1 (Markdown):

MARKDOWN

# Weather Data Analysis

Exploring a simulated dataset of 100 days of weather measurements.

Cell 2 (Code): Create the dataset (code above).

Cell 3 (Code):

PYTHON

print("First 5 rows of the dataset:")
print(df.head())

Cell 4 (Code):

PYTHON

print("\nDataFrame Information:")
df.info()

Cell 5 (Code):

PYTHON

mean_temp = df['temperature'].mean()
mean_humidity = df['humidity'].mean()
print(f"Mean temperature: {mean_temp:.2f} F")
print(f"Mean humidity: {mean_humidity:.2f}%")

Expected output includes a nicely formatted table with dates, temperatures (around 65F), humidity (around 60%), and wind speeds, followed by calculated means.

Key Points

• Python code runs normally in Jupyter cells, with enhanced display capabilities
• NumPy, pandas, and matplotlib are the core data science libraries
• pandas can load data from CSV, Excel, JSON, HDF5, and many other formats
• Magic commands (% and %%) add special IPython functionality like timing code
• Use print() or display() to show multiple outputs from a single cell