Broadcasting — Multi-dimensional Array Operation Rules

After completing this topic, you will:

Understand why NumPy broadcasting exists, be able to determine whether operations are possible by applying the three rules, and be able to write efficient code using broadcasting.

Why is Broadcasting Necessary?

python

import numpy as np

# I want to add 10 to every element
arr = np.array([1, 2, 3, 4, 5])
result = arr + 10
print(result)  # [11 12 13 14 15]

arr has 5 elements, and 10 is a scalar (1 element). Although the sizes are different, addition is possible. This is because NumPy automatically expands 10 to [10, 10, 10, 10, 10]. This automatic expansion is broadcasting.

Without broadcasting:

python

# You would have to write something like this every time
result = arr + np.full(5, 10)  # Inefficient

Operations with the Same Shape — The Basics

If the sizes are the same, it's an element-wise operation.

python

a = np.array([1, 2, 3])
b = np.array([10, 20, 30])

print(a + b)   # [11 22 33]
print(a * b)   # [10 40 90]

The same applies to 2D arrays.

python

A = np.array([[1, 2], [3, 4]])
B = np.array([[10, 20], [30, 40]])

print(A + B)
# [[11 22]
#  [33 44]]

Broadcasting Rules — 3 Rules

When the sizes are different, NumPy applies the three rules in order.

Rule 1: If the number of dimensions is different, add a 1 to the beginning of the smaller array.

python

a = np.array([[1, 2, 3],    # shape: (2, 3)
              [4, 5, 6]])
b = np.array([10, 20, 30])  # shape: (3,)

# Rule 1: Expand b's shape (3,) to (1, 3)
# After applying Rule 2: (1, 3) -> (2, 3)
print(a + b)
# [[11 22 33]
#  [14 25 36]]

Rule 2: Expand the dimensions of size 1 to match the other array.

python

a = np.array([[1, 2, 3]])   # shape: (1, 3)
b = np.array([[10],          # shape: (3, 1)
              [20],
              [30]])

# a: (1, 3) -> Expand along the row direction -> (3, 3)
# b: (3, 1) -> Expand along the column direction -> (3, 3)
print(a + b)
# [[11 12 13]
#  [21 22 23]
#  [31 32 33]]

Rule 3: If the sizes are different and neither side is 1, then it's an error.

python

a = np.array([1, 2, 3])     # shape: (3,)
b = np.array([10, 20])      # shape: (2,)

# 3 vs 2 — Neither side is 1 -> Error!
# a + b  -> ValueError: operands could not be broadcast together

Understanding Visually

text

(4, 3) + (3,)    -> OK!
a:  [[1 2 3]      b: [10 20 30]
     [4 5 6]          ↓ Rule 1: (1, 3)
     [7 8 9]          ↓ Rule 2: (4, 3)
     [0 1 2]]
                  b': [[10 20 30]
                       [10 20 30]
                       [10 20 30]
                       [10 20 30]]

Result: [[11 22 33]
         [14 25 36]
         [17 28 39]
         [10 21 32]]

text

(3, 1) + (1, 4)  -> OK!  Result shape: (3, 4)
a: [[1]           b: [[10 20 30 40]]
    [2]               ↓ Rule 2: (3, 4)
    [3]]          b': [[10 20 30 40]
    ↓ Rule 2           [10 20 30 40]
a': [[1 1 1 1]        [10 20 30 40]]
     [2 2 2 2]
     [3 3 3 3]]

Result: [[11 12 13 14]
         [12 13 14 15]
         [13 14 15 16]]

Practical Pattern — Normalization

python

# Center by subtracting the mean of each column
data = np.array([[170, 60, 30],
                 [180, 75, 25],
                 [165, 55, 35],
                 [175, 70, 28]])
# shape: (4, 3) — 4 people, 3 measurements (height, weight, age)

col_mean = data.mean(axis=0)  # shape: (3,) — Mean of each column
print(col_mean)  # [172.5  65.   29.5]

centered = data - col_mean    # (4, 3) - (3,) -> Broadcasting!
print(centered)
# [[ -2.5  -5.    0.5]
#  [  7.5  10.   -4.5]
#  [ -7.5 -10.    5.5]
#  [  2.5   5.   -1.5]]

Z-score Normalization

python

col_std = data.std(axis=0)
z_scores = (data - col_mean) / col_std  # Broadcasting 2 times!

(4, 3) - (3,) -> OK, (4, 3) / (3,) -> OK. This single pattern can normalize the entire dataset.

Broadcasting vs. Loops

python

# Bad: Python loop — Slow
data = np.random.rand(10000, 100)
mean = data.mean(axis=0)

result = np.zeros_like(data)
for i in range(data.shape[0]):
    for j in range(data.shape[1]):
        result[i, j] = data[i, j] - mean[j]

# Good: Broadcasting — 100x+ Faster
result = data - mean

Broadcasting is a vector operation implemented in C internally. It's tens to hundreds of times faster than Python loops, and the code is only one line.

Practical Pattern — Distance Matrix

Calculate all distances between two sets of points at once.

python

# 3 points: (1, 0), (2, 3), (4, 1)
points = np.array([[1, 0], [2, 3], [4, 1]])  # shape: (3, 2)

# Euclidean distance between all pairs
diff = points[:, np.newaxis, :] - points[np.newaxis, :, :]
# (3, 1, 2) - (1, 3, 2) -> Broadcasting -> (3, 3, 2)

dist = np.sqrt((diff ** 2).sum(axis=2))
print(dist.round(2))
# [[0.   3.16 3.16]
#  [3.16 0.   2.83]
#  [3.16 2.83 0.  ]]

Use np.newaxis to add dimensions and calculate the differences between all pairs at once using broadcasting. This is tens of times faster than a double for loop.

Caution — Memory Explosion

python

a = np.random.rand(10000, 1)     # shape: (10000, 1)
b = np.random.rand(1, 10000)     # shape: (1, 10000)
c = a + b                         # shape: (10000, 10000) — 100 million elements!
# Memory: 10000 × 10000 × 8 bytes = 800 MB

Broadcasting expands conceptually, but the resulting array actually occupies memory. (10000, 1) + (1, 10000) creates an array of size (10000, 10000). With large data, calculate the resulting shape in advance to check the memory usage.

python

# Predict memory usage
result_shape = (10000, 10000)
dtype_size = 8  # float64 = 8 bytes
memory_bytes = result_shape[0] * result_shape[1] * dtype_size
print(f"{memory_bytes / 1e6:.0f} MB")  # 800 MB

Quick Shape Compatibility Check

Match from the right, and if each dimension is the same or one is 1, it's compatible.

text

(4, 3) + (3,)      -> (4, 3) + (1, 3) -> OK  -> (4, 3)
(4, 3) + (4, 1)    -> OK  -> (4, 3)
(3, 1) + (1, 4)    -> OK  -> (3, 4)
(4, 3) + (2,)      -> (4, 3) + (1, 2) -> 3 vs 2 -> ERROR
(2, 3, 4) + (3, 1) -> (2, 3, 4) + (1, 3, 1) -> OK -> (2, 3, 4)

Key Takeaways

Rule	Description
Rule 1	If the number of dimensions is different, add a 1 to the beginning of the smaller array.
Rule 2	Expand the dimensions of size 1 to match the other array.
Rule 3	If the sizes are different and neither side is 1, then it's an error.

Broadcasting is one of NumPy's most powerful features. It allows you to perform data normalization, distance calculations, matrix transformations—all without for loops. The key is to look at the shape and instantly judge "match from the right, and if it's the same or 1, it's OK".

The same principle applies in pandas. Operations like DataFrame - Series are broadcast along the columns. Understanding NumPy broadcasting will naturally lead to an understanding of vector operations in pandas.

python

import pandas as pd

df = pd.DataFrame({"A": [10, 20, 30], "B": [40, 50, 60]})
means = df.mean()      # Series: A=20, B=50
centered = df - means   # Broadcasting! Subtract the mean from each column
print(centered)
#      A     B
# 0 -10.0 -10.0
# 1   0.0   0.0
# 2  10.0  10.0

BioPlayground

Broadcasting — Rules for Multidimensional Array Operations

Broadcasting — Multi-dimensional Array Operation Rules

After completing this topic, you will:

Why is Broadcasting Necessary?

Operations with the Same Shape — The Basics

Broadcasting Rules — 3 Rules

Rule 1: If the number of dimensions is different, add a 1 to the beginning of the smaller array.

Rule 2: Expand the dimensions of size 1 to match the other array.

Rule 3: If the sizes are different and neither side is 1, then it's an error.

Understanding Visually

Practical Pattern — Normalization

Z-score Normalization

Broadcasting vs. Loops

Practical Pattern — Distance Matrix

Caution — Memory Explosion

Quick Shape Compatibility Check

Key Takeaways

💬 Questions & Comments