Reading and Writing Experiment Data Files with Python

After Completing This Topic

You'll be able to read text files and CSV files in Python, filter data, and save results to new files.

Why File I/O Matters

Experiment data is ultimately files. You export from instruments as CSV, save analysis results as text, and share files with other researchers. You could open them in Excel, but when you have thousands of samples, doing it by hand isn't practical.

With Python file handling — extracting only the samples that meet your criteria from a 10,000-line CSV takes just 5 lines of code.

Reading Text Files: open()

This is the most basic way to read files in Python.

with open("protocol.txt", "r", encoding="utf-8") as f:
    content = f.read()

print(content)

Key concepts:

• open("filename", "mode") — function that opens a file
• "r" — read mode
• encoding="utf-8" — prevents character encoding issues
• with ... as f: — safely opens the file and automatically closes it when the block ends

Without with, you'd have to call f.close() manually. Just as you turn off equipment after an experiment, files need to be closed after use. with does this automatically.

File Mode Reference

Be careful with "w" mode — existing file contents are completely erased. If you want to preserve existing data and add more, use "a" mode.

Reading Line by Line

When you want to process a file one line at a time:

with open("samples.txt", "r", encoding="utf-8") as f:
    for line in f:
        line = line.strip()
        if line:
            print(line)

strip() removes the \n (newline character) at the end of each line. Without it, blank lines appear in the output.

Reading CSV Files

CSV (Comma-Separated Values) is one of the standard formats for experiment data. Data exported from instruments, TCGA clinical data, data downloaded from NCBI — all come as CSV.

sample_id,gene,expression,status
S001,EGFR,12.5,high
S002,TP53,3.2,low
S003,EGFR,18.7,high
S004,BRCA1,7.1,medium
S005,TP53,2.8,low

Read it with Python's csv module:

import csv

with open("expression_data.csv", "r", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(f"{row['sample_id']}: {row['gene']} = {row['expression']}")

csv.DictReader uses the first line (header) as keys, converting each row into a dictionary. This lets you access columns by name like row['gene'], which is very convenient.

Conditional Filtering

"I want to extract only the EGFR gene" — just combine a loop with a conditional:

import csv

egfr_samples = []

with open("expression_data.csv", "r", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        if row["gene"] == "EGFR":
            egfr_samples.append(row)

print(f"EGFR samples: {len(egfr_samples)}")
for sample in egfr_samples:
    print(f"  {sample['sample_id']}: expression = {sample['expression']}")

assert len(egfr_samples) == 2

Notice how for, if, append, and f-string from previous topics all come together here. This is how coding works — what you've learned accumulates and combines.

Writing CSV Files

Save filtered results to a new file:

import csv

high_expression = [
    {"sample_id": "S001", "gene": "EGFR", "expression": "12.5"},
    {"sample_id": "S003", "gene": "EGFR", "expression": "18.7"},
]

with open("egfr_high.csv", "w", encoding="utf-8", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["sample_id", "gene", "expression"])
    writer.writeheader()
    writer.writerows(high_expression)

print("egfr_high.csv saved successfully")

newline="" prevents extra blank lines on Windows. It's a good habit to include it on Mac/Linux too.

Working with TSV Files

TSV (Tab-Separated Values) uses tabs (\t) as delimiters. In bioinformatics, TSV is used more often than CSV — NCBI GEO data, BED files, DEG results, etc.

import csv

with open("deg_results.tsv", "r", encoding="utf-8") as f:
    reader = csv.DictReader(f, delimiter="\t")
    for row in reader:
        log2fc = float(row["log2FoldChange"])
        pvalue = float(row["pvalue"])
        if abs(log2fc) > 1.0 and pvalue < 0.05:
            print(f"{row['gene']}: log2FC={log2fc:.2f}, p={pvalue:.4f}")

Just adding delimiter="\t" lets the same CSV code work for TSV files.

Try It Yourself (Faded Example)

Fill in the blanks to complete a code that filters samples meeting specific criteria from a CSV file.

import 
with open("samples.csv", "r", encoding="utf-8") as f:
  reader = csv.Reader(f)
  for row in reader:
      od = float(row[""])
      if od > 1.0:
          print(f"{row['name']}: OD = {od}")

Common Errors & Solutions

Q: I get FileNotFoundError: No such file or directory

The file path is wrong. Python resolves relative paths from the directory where the script was executed. Use ls (terminal) to check if the file is in the current directory. In Google Colab, upload the file through the file tab on the left.

Q: Numbers in my CSV are read as strings

csv.DictReader reads all values as strings. If you need numeric operations, convert with float(row["column"]) or int(row["column"]).

Q: Characters are garbled

Either you forgot encoding="utf-8", or the file was saved with a different encoding (e.g., euc-kr). When saving CSV from Korean Excel, it often defaults to euc-kr. Try changing to encoding="euc-kr".

Q: What happens if I don't use with in open()?

You can open with f = open("file.txt"), but if the program crashes with an error, the file won't be closed. with automatically closes it even when errors occur. Like turning off equipment after an experiment — with handles it automatically.