What is a DataFrame?

A DataFrame is a two-dimensional labeled data structure with columns of potentially different types. Think of it as a spreadsheet or SQL table. It's the primary data structure in Pandas for data analysis.

Creating DataFrames

From Dictionary

import pandas as pd

data = {
    'name': ['Alice', 'Bob', 'Charlie'],
    'age': [25, 30, 35],
    'city': ['NYC', 'LA', 'Chicago']
}

df = pd.DataFrame(data)
print(df)
#       name  age     city
# 0    Alice   25      NYC
# 1      Bob   30       LA
# 2  Charlie   35  Chicago

From List of Dictionaries

data = [
    {'name': 'Alice', 'age': 25, 'city': 'NYC'},
    {'name': 'Bob', 'age': 30, 'city': 'LA'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'}
]

df = pd.DataFrame(data)

From CSV File

df = pd.read_csv('data.csv')
df = pd.read_csv('data.csv', index_col=0)  # Use first column as index

Viewing Data

Inspect your DataFrame:

# First few rows
df.head()                    # First 5 rows
df.head(10)                  # First 10 rows

# Last few rows
df.tail()                    # Last 5 rows

# Basic information
df.info()                     # Data types and non-null counts
df.describe()                 # Statistical summary
df.shape                      # (rows, columns)
df.columns                    # Column names
df.index                      # Row indices

Selecting Data

Select Columns

# Single column (returns Series)
ages = df['age']

# Multiple columns (returns DataFrame)
subset = df[['name', 'age']]

# Using dot notation (only for valid Python identifiers)
ages = df.age

Select Rows

# By index
first_row = df.iloc[0]        # First row
first_three = df.iloc[0:3]    # First 3 rows

# By label
row = df.loc[0]               # Row with index 0

# By condition
adults = df[df['age'] >= 18]
nyc_people = df[df['city'] == 'NYC']

Filtering Data

Filter rows based on conditions:

# Single condition
adults = df[df['age'] >= 25]

# Multiple conditions (use & for AND, | for OR)
young_adults = df[(df['age'] >= 18) & (df['age'] < 30)]
nyc_or_la = df[(df['city'] == 'NYC') | (df['city'] == 'LA')]

# Using query() method
adults = df.query('age >= 25')
nyc_people = df.query("city == 'NYC'")

Adding and Modifying Columns

# Add new column
df['salary'] = [50000, 60000, 70000]

# Modify existing column
df['age'] = df['age'] + 1     # Add 1 to all ages

# Create column from calculation
df['age_next_year'] = df['age'] + 1

# Using assign() (returns new DataFrame)
df_new = df.assign(age_next_year=df['age'] + 1)

Basic Operations

Sorting

# Sort by column
df_sorted = df.sort_values('age')
df_sorted = df.sort_values('age', ascending=False)  # Descending

# Sort by multiple columns
df_sorted = df.sort_values(['city', 'age'])

Aggregations

# Basic statistics
df['age'].mean()              # Average age
df['age'].sum()               # Sum of ages
df['age'].max()               # Maximum age
df['age'].min()               # Minimum age
df['age'].count()             # Count non-null values

# Multiple aggregations
df.describe()                 # Summary statistics

Handling Missing Data

# Check for missing values
df.isnull()                   # Boolean DataFrame
df.isnull().sum()             # Count missing per column

# Drop rows with missing values
df_clean = df.dropna()

# Fill missing values
df_filled = df.fillna(0)      # Fill with 0
df_filled = df.fillna(df.mean())  # Fill with mean

Grouping Data

# Group by column
grouped = df.groupby('city')

# Aggregate after grouping
avg_age_by_city = df.groupby('city')['age'].mean()
count_by_city = df.groupby('city').size()

# Multiple aggregations
summary = df.groupby('city').agg({
    'age': ['mean', 'min', 'max'],
    'salary': 'sum'
})

Common Operations

Rename Columns

df = df.rename(columns={'name': 'full_name', 'age': 'years'})

Drop Columns

df = df.drop('city', axis=1)           # Drop one column
df = df.drop(['city', 'salary'], axis=1)  # Drop multiple columns

Drop Rows

df = df.drop(0)                       # Drop row with index 0
df = df.drop([0, 1])                  # Drop multiple rows

Best Practices

  • Use iloc for position-based indexing: When you know the position
  • Use loc for label-based indexing: When you know the label
  • Check data first: Always use head(), info(), and describe() to understand your data
  • Handle missing data: Decide whether to drop or fill missing values
  • Use vectorized operations: Faster than loops

Next Steps

Learn about filtering data in detail, or explore GroupBy for advanced aggregations.