Querying Biology Databases via API

After Completing This Topic

You'll be able to call NCBI and UniProt APIs using Python's requests library, extract needed data from JSON responses, and query multiple genes in a loop.

What Is an API?

You've probably searched for genes on the NCBI website before. Type "EGFR" into the search bar and a gene information page appears. Behind the scenes, your browser sends a request to the NCBI server, and the server sends back a response.

API (Application Programming Interface) is doing this same process with code. Instead of a browser, Python sends the request, and instead of HTML, you receive structured data (JSON) as the response.

Lab analogy — just as you insert a sample and press a button to get results from an instrument, you put a gene name into an API and get information out. The only difference is that code presses the button automatically instead of a person.

The requests Library

pip install requests

import requests

response = requests.get("https://api.github.com")
print(f"Status code: {response.status_code}")
print(f"Response data: {response.json()}")

requests.get(URL) — sends a GET request to the URL. It's the same as entering a URL in a web browser.

Status codes:

• 200 — Success
• 404 — URL not found
• 429 — Too many requests (rate limit)
• 500 — Internal server error

JSON: The API Response Format

Most APIs respond in JSON format. It's almost identical in structure to a Python dictionary:

{
  "gene": "EGFR",
  "organism": "Homo sapiens",
  "chromosome": "7",
  "aliases": ["ERBB1", "HER1"]
}

Working with JSON responses in Python:

import requests

response = requests.get("https://rest.uniprot.org/uniprotkb/P04637.json")
data = response.json()

print(f"Protein: {data['proteinDescription']['recommendedName']['fullName']['value']}")
print(f"Organism: {data['organism']['scientificName']}")
print(f"Sequence length: {data['sequence']['length']}")

response.json() — converts the JSON response to a Python dictionary. Then access values with data["key"].

Practical: Fetching Protein Info from UniProt API

import requests

def get_protein_info(uniprot_id: str) -> dict:
    url = f"https://rest.uniprot.org/uniprotkb/{uniprot_id}.json"
    response = requests.get(url)

    if response.status_code != 200:
        print(f"Error: {uniprot_id} — status code {response.status_code}")
        return {}

    data = response.json()
    return {
        "id": uniprot_id,
        "name": data["proteinDescription"]["recommendedName"]["fullName"]["value"],
        "organism": data["organism"]["scientificName"],
        "length": data["sequence"]["length"],
    }

info = get_protein_info("P04637")
print(info)

By wrapping it in a function, you can query multiple proteins in a loop:

ids = ["P04637", "P00533", "P38398"]
results = []

for uid in ids:
    info = get_protein_info(uid)
    if info:
        results.append(info)
        print(f"  ✓ {info['name']} ({info['length']} aa)")

print(f"\nTotal {len(results)} queries completed")

Practical: NCBI E-utilities

NCBI provides an API called E-utilities. You can search genes, download sequences, and search publications — all with code.

import requests

gene_name = "BRCA1"
url = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi"
params = {
    "db": "gene",
    "term": f"{gene_name}[Gene Name] AND Homo sapiens[Organism]",
    "retmode": "json",
}

response = requests.get(url, params=params)
data = response.json()

gene_ids = data["esearchresult"]["idlist"]
print(f"{gene_name} search results: {gene_ids}")

params — passes URL parameters as a dictionary. requests automatically appends ?db=gene&term=... to the URL.

API Etiquette: Rate Limiting

APIs can't be called infinitely. You need intervals between requests to avoid overloading the server:

import time
import requests

ids = ["P04637", "P00533", "P38398", "Q13315", "P42336"]

for uid in ids:
    info = get_protein_info(uid)
    if info:
        print(f"  {info['name']}")
    time.sleep(0.5)

time.sleep(0.5) — wait 0.5 seconds. NCBI recommends about 3 requests per second, UniProt about 10. This is like shared equipment etiquette in the lab — if one person monopolizes it, nobody else can use it.

API Keys

Some APIs require an API key. NCBI E-utilities works without one, but registering a key increases the rate limit from 3 to 10 requests per second.

params = {
    "db": "gene",
    "term": "TP53[Gene Name]",
    "retmode": "json",
    "api_key": "YOUR_API_KEY_HERE",
}

Never put API keys directly in your code. Store them in environment variables or separate config files, and don't commit them to Git.

Error Handling

API calls can fail due to network issues or invalid IDs:

import requests

def safe_api_call(url: str, params: dict = None) -> dict:
    try:
        response = requests.get(url, params=params, timeout=10)
        response.raise_for_status()
        return response.json()
    except requests.exceptions.Timeout:
        print("Request timed out — try again later")
        return {}
    except requests.exceptions.HTTPError as e:
        print(f"HTTP error: {e}")
        return {}

timeout=10 — give up if no response within 10 seconds. Prevents the program from hanging forever when the server is slow.

Try It Yourself (Faded Example)

Fill in the blanks to complete a code that fetches protein sequence length from the UniProt API.

import 
url = "https://rest.uniprot.org/uniprotkb/P04637.json"
response = requests.(url)
if response.status_code == :
  data = response.()
  length = data["sequence"]["length"]
  print(f"Sequence length: {length} aa")

Common Errors & Solutions

Q: I get ConnectionError or Timeout

Check your internet connection. The server might be temporarily down. Try again after time.sleep(5), or run it later.

Q: KeyError — can't get values from JSON

The API response structure might differ from your expectation. Print the full response with print(json.dumps(data, indent=2)) to check the actual key structure. Or use data.get("key", "not found") to return a default when the key doesn't exist.

Q: API response comes as HTML

The URL might be a webpage address (for browsers), not an API address. Check the API documentation for the correct endpoint. URLs containing /api/ or .json are typically API endpoints.

Q: 429 Too Many Requests error

You're sending too many requests too quickly. Increase the interval with time.sleep(1). NCBI recommends registering an API key.