apply, map, applymap: Three Tools for Data Transformation

After completing this topic, you will:

Clearly distinguish between the three functions apply, map, and applymap.
Choose the appropriate tool for the task at hand.
Understand the performance implications of each.

Why Have Three in the First Place?

It can be confusing that pandas has three functions for transforming data. Each has a different scope of application.

Function	Target	Unit of Application
`map()`	Series (1-dimensional)	Individual values
`apply()`	Series or DataFrame	Row or column-wise
`applymap()`	DataFrame (2-dimensional)	Individual values

In a nutshell: map applies a function to each value in a Series, applymap applies a function to each value in a DataFrame, and apply applies a function to entire rows or columns.

map: Transforming Each Value in a Series

map() is specifically for Series. It applies a function to each value or maps values using a dictionary.

Function Mapping

python

import pandas as pd

df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol"],
    "score": [85.7, 92.3, 78.1]
})

# Convert each name to uppercase
df["name"].map(str.upper)
# 0    ALICE
# 1      BOB
# 2    CAROL

# Round each score
df["score"].map(round)
# 0    86
# 1    92
# 2    78

Dictionary Mapping

python

grade_map = {
    "A": "Excellent",
    "B": "Good",
    "C": "Average"
}

grades = pd.Series(["A", "B", "C", "A", "B"])
grades.map(grade_map)
# 0    Excellent
# 1         Good
# 2      Average
# 3    Excellent
# 4         Good

Values not in the dictionary will become NaN. This is commonly used for category encoding in practice.

With Lambda

python

df["score"].map(lambda x: "Pass" if x >= 80 else "Fail")
# 0    Pass
# 1    Pass
# 2    Fail

apply: Applying a Function Row- or Column-wise

apply() can be used with both Series and DataFrames.

Series.apply: Similar to map

python

df["score"].apply(lambda x: round(x, 1))
# Produces the same result as map()

For Series, it's almost identical to map(). The difference is that apply() can take additional arguments.

DataFrame.apply: Column-wise (or Row-wise)

python

df = pd.DataFrame({
    "math": [85, 92, 78],
    "english": [90, 88, 95],
    "science": [88, 91, 82]
})

# Average of each subject (column)
df.apply("mean")           # axis=0 (default, column direction)
# math       85.0
# english    91.0
# science    87.0

# Average of each student (row)
df.apply("mean", axis=1)   # axis=1 (row direction)
# 0    87.666667
# 1    90.333333
# 2    85.000000

axis=0 means "top to bottom" (apply the function to each column), and axis=1 means "left to right" (apply the function to each row).

Referencing Multiple Columns in Row-wise Application

python

df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol"],
    "math": [85, 92, 78],
    "english": [90, 88, 95]
})

# Each student's higher score of the two subjects
df.apply(lambda row: max(row["math"], row["english"]), axis=1)
# 0    90
# 1    92
# 2    95

In row-wise (axis=1) application, row is a Series representing that row. You can reference multiple columns to perform complex calculations.

applymap: Transforming Every Value in a DataFrame

applymap() is specifically for DataFrames and applies a function to every cell.

python

df = pd.DataFrame({
    "math": [85.7, 92.3, 78.1],
    "english": [90.2, 88.9, 95.4],
    "science": [88.1, 91.7, 82.3]
})

# Convert all values to integers
df.applymap(int)
#    math  english  science
# 0    85       90       88
# 1    92       88       91
# 2    78       95       82

# Apply a format to all values
df.applymap(lambda x: f"{x:.1f}%")
#     math english science
# 0  85.7%   90.2%   88.1%
# 1  92.3%   88.9%   91.7%
# 2  78.1%   95.4%   82.3%

Note: In pandas 2.1+, applymap() has been merged into map(). You can achieve the same result using DataFrame.map(). However, many codebases still use applymap(), so it's important to be aware of it.

Comparing the Three Functions: At a Glance

python

import pandas as pd

df = pd.DataFrame({
    "A": [1, 2, 3],
    "B": [4, 5, 6]
})

# map: Each value in a Series → value
df["A"].map(lambda x: x * 10)        # Series → Series
# 0    10
# 1    20
# 2    30

# apply (Series): Similar to map
df["A"].apply(lambda x: x * 10)      # Series → Series (same result)

# apply (DataFrame, axis=0): Column-wise
df.apply(sum)                          # DataFrame → Series
# A     6
# B    15

# apply (DataFrame, axis=1): Row-wise
df.apply(sum, axis=1)                  # DataFrame → Series
# 0     5
# 1     7
# 2     9

# applymap: Each value in a DataFrame → value
df.applymap(lambda x: x * 10)         # DataFrame → DataFrame
#     A   B
# 0  10  40
# 1  20  50
# 2  30  60

Performance: Prioritize Vectorized Operations

apply, map, and applymap internally use Python loops. Pandas' built-in operations (vectorized operations) are much faster.

python

import numpy as np

df = pd.DataFrame({"value": range(1_000_000)})

# Slow — apply (Python loop)
%timeit df["value"].apply(lambda x: x * 2)
# ~200ms

# Fast — vectorized operation (executed in C)
%timeit df["value"] * 2
# ~2ms (100x faster)

Rule: Simple arithmetic, comparisons, and string methods should use vectorized operations. Use apply() only for complex logic that cannot be expressed as a vectorized operation.

python

# Bad — use apply for conditional branching
df["label"] = df["value"].apply(lambda x: "high" if x > 500000 else "low")

# Good — np.where (vectorized operation)
df["label"] = np.where(df["value"] > 500000, "high", "low")

# Bad — use apply for string manipulation
df["upper"] = df["name"].apply(str.upper)

# Good — str accessor (vectorized operation)
df["upper"] = df["name"].str.upper()

Practical Patterns: Complex Transformations

Use apply when vectorized operations are not possible:

python

df = pd.DataFrame({
    "name": ["Alice Smith", "Bob Lee", "Carol Park"],
    "birth": ["1995-03-15", "1988-11-22", "2001-07-08"],
    "department": ["Sales", "Dev", "HR"]
})

def create_employee_id(row):
    dept_code = row["department"][:2].upper()
    last_name = row["name"].split()[-1].upper()
    year = row["birth"][:4]
    return f"{dept_code}-{last_name}-{year}"

df["emp_id"] = df.apply(create_employee_id, axis=1)
print(df["emp_id"])
# 0    SA-SMITH-1995
# 1      DE-LEE-1988
# 2     HR-PARK-2001

Creating complex strings by referencing multiple columns – this is difficult to express with vectorized operations, and apply(axis=1) is appropriate.

Key Takeaways

Scenario	Tool
Transform each value in a Series	`map()` or `apply()`
Map values using a dictionary	`map(dict)`
Transform each cell in a DataFrame	`applymap()` (or pandas 2.1+ `map()`)
Aggregate column-wise	`apply(func, axis=0)`
Complex row-wise calculations	`apply(func, axis=1)`
Simple arithmetic/comparisons	Vectorized operations (don't use apply)

The order to choose the functions: 1) Can it be done with vectorized operations? → vectorized operations. 2) Each value in a Series? → map. 3) Each cell in a DataFrame? → applymap. 4) Complex row/column-wise logic? → apply. Remembering this order will ensure both performance and readability.

Common Mistakes

Mistake	Problem	Solution
`df.map(func)` (pandas < 2.1)	`AttributeError`	`df.applymap(func)` or upgrade pandas
`df.apply(func)` returns a scalar	Unexpected result	Check axis – 0 is column, 1 is row
Use apply for simple arithmetic	100x slower	Replace with vectorized operations
Missing keys in map	NaN values	Use `fillna()` to specify a default value

BioPlayground

apply, map, applymap — Three Tools for Data Transformation

apply, map, applymap: Three Tools for Data Transformation

After completing this topic, you will:

Why Have Three in the First Place?

map: Transforming Each Value in a Series

Function Mapping

Dictionary Mapping

With Lambda

apply: Applying a Function Row- or Column-wise

Series.apply: Similar to map

DataFrame.apply: Column-wise (or Row-wise)

Referencing Multiple Columns in Row-wise Application

applymap: Transforming Every Value in a DataFrame

Comparing the Three Functions: At a Glance

Performance: Prioritize Vectorized Operations

Practical Patterns: Complex Transformations

Key Takeaways

Common Mistakes

💬 Questions & Comments