Reading CSV Files

The most common file format for data:

import pandas as pd

# Basic CSV reading
df = pd.read_csv('data.csv')

# With options
df = pd.read_csv('data.csv', 
                 sep=',',           # Delimiter
                 header=0,          # Row to use as column names
                 index_col=0,       # Column to use as index
                 skiprows=1,        # Skip first row
                 nrows=100)          # Read only first 100 rows

Reading Excel Files

Read from Excel spreadsheets:

# Read Excel file
df = pd.read_excel('data.xlsx')

# Read specific sheet
df = pd.read_excel('data.xlsx', sheet_name='Sheet1')

# Read multiple sheets
sheets = pd.read_excel('data.xlsx', sheet_name=['Sheet1', 'Sheet2'])

# Read specific columns
df = pd.read_excel('data.xlsx', usecols=['Name', 'Age'])

Reading JSON Files

Read JSON data:

# Read JSON
df = pd.read_json('data.json')

# From JSON string
json_string = '{"name": ["Alice", "Bob"], "age": [25, 30]}'
df = pd.read_json(json_string)

# Nested JSON
df = pd.json_normalize(data)  # Flatten nested structures

Reading from URLs

Read data directly from web:

# Read CSV from URL
url = 'https://example.com/data.csv'
df = pd.read_csv(url)

# Read JSON from API
df = pd.read_json('https://api.example.com/data')

Handling Different Encodings

Specify encoding for special characters:

# UTF-8 (default)
df = pd.read_csv('data.csv', encoding='utf-8')

# Latin-1
df = pd.read_csv('data.csv', encoding='latin-1')

# Auto-detect
df = pd.read_csv('data.csv', encoding_errors='ignore')

Handling Missing Data

Control how missing values are handled:

# Specify missing value markers
df = pd.read_csv('data.csv', na_values=['N/A', 'NULL', ''])

# Keep empty strings
df = pd.read_csv('data.csv', keep_default_na=False)

Reading Large Files

Optimize for large datasets:

# Read in chunks
chunk_size = 10000
for chunk in pd.read_csv('large_file.csv', chunksize=chunk_size):
    process(chunk)

# Specify data types (faster)
dtypes = {'name': 'str', 'age': 'int32', 'price': 'float32'}
df = pd.read_csv('data.csv', dtype=dtypes)

Common Options

Useful parameters:

df = pd.read_csv('data.csv',
    sep=',',              # Delimiter
    header=0,              # Header row
    names=['col1', 'col2'],  # Custom column names
    skiprows=[0, 2],       # Skip specific rows
    skipfooter=1,          # Skip last N rows
    nrows=1000,            # Read only N rows
    usecols=[0, 2, 4],     # Read specific columns
    parse_dates=['date'],  # Parse dates
    thousands=',',         # Thousands separator
    decimal='.')           # Decimal separator

Best Practices

  • Check file encoding: Use appropriate encoding parameter
  • Specify dtypes: Faster loading for large files
  • Use chunksize: For files larger than memory
  • Handle missing data: Specify na_values appropriately
  • Validate data: Check shape and dtypes after reading

Next Steps

Learn about working with DataFrames or filtering data.