Biological Databases and APIs

Before you start

You have seen where biological data is stored in Where the data lives.
Any term new? The Glossary has it.

Learning objectives

By the end of this lesson you will be able to: name the major databases (NCBI, Ensembl, UniProt) and what each holds, recognise the common gene ID types (symbol, Ensembl ENSG, Entrez/NCBI gene ID, RefSeq NM_), explain why ID mapping matters, and fetch records politely with an API while respecting etiquette.

Where the data lives

Public bioinformatics data is spread across a few large hubs, each with a focus. You rarely download whole databases; instead you query them through an API (an application programming interface), which lets your script ask for exactly the records you need.

Database	Holds	How to access
NCBI	GenBank and RefSeq sequences, SRA reads, dbSNP variants, PubMed	Entrez E-utilities
Ensembl	Genes, transcripts, annotation, comparative genomics	BioMart and a REST API
UniProt	Protein sequences and functional annotation	REST API and a mapping tool

RefSeq vs GenBank

GenBank is the raw submitted record; RefSeq is NCBI's curated, non-redundant reference set (RefSeq transcript IDs start with NM_ for mRNAs). When you want a single canonical record per gene, you usually want RefSeq.

1ID types and why mapping matters

The same gene is named differently in different systems, and tools only join data when the IDs match exactly. A gene symbol is human-friendly but ambiguous and changes over time; the stable database IDs are what you should key your analysis on.

ID type	Example	Source
Gene symbol	TP53	HGNC (human-friendly, can change)
Ensembl gene ID	ENSG00000141510	Ensembl (stable)
Entrez / NCBI gene ID	7157	NCBI (a number)
RefSeq transcript	NM_000546	NCBI RefSeq
UniProt accession	P04637	UniProt (protein)

Joining two datasets that use different ID systems requires mapping one to the other. Common tools are Ensembl BioMart and, in R, the org.Hs.eg.db annotation package.

A typical mapping path: a gene symbol resolves to a stable Ensembl gene ID, which maps to a UniProt protein accession.

2Fetching politely: API etiquette

Public APIs are a shared resource. Good citizenship keeps them fast for everyone and keeps you from being blocked.

Identify yourself: for NCBI Entrez, always set Entrez.email (and an API key for higher limits).
Respect rate limits: without a key, NCBI allows only a few requests per second, so batch your queries instead of looping one ID at a time.
Cache results: save what you download so a re-run does not hit the server again.

3Worked example: Biopython Entrez efetch

This snippet (shown for illustration, not meant to be run as-is) fetches a nucleotide record from NCBI with Biopython. Note the email set up front, which Entrez requires.

# illustration only -- set a real email before running
from Bio import Entrez, SeqIO

Entrez.email = "you@example.com"   # required etiquette

handle = Entrez.efetch(
    db="nucleotide",
    id="NM_000546",        # RefSeq mRNA for TP53
    rettype="gb",
    retmode="text",
)
record = SeqIO.read(handle, "genbank")
handle.close()
print(record.id, len(record.seq))

Why set Entrez.email

If a query misbehaves, NCBI uses the email to contact you before blocking your traffic. Leaving it unset is both impolite and a fast way to get rate-limited.

Check your understanding

Which database is the place to look for protein sequences and functional annotation?

Right. UniProt is the protein resource. dbSNP holds variants and SRA holds raw sequencing reads, both at NCBI.

Which of these is an Ensembl gene ID?

Correct. Ensembl gene IDs start with ENSG. NM_ is a RefSeq transcript; 7157 is an Entrez/NCBI gene ID.

Why should you avoid keying an analysis on gene symbols alone?

Exactly. Gene symbols get renamed and can be ambiguous. Stable IDs like ENSG or Entrez gene IDs are safer keys for joining data.

When using NCBI Entrez via Biopython, what must you set before fetching?

Right. NCBI requires Entrez.email so they can reach you before blocking misbehaving traffic. An API key raises your rate limit further.

Your dataset uses gene symbols and another uses Entrez IDs. What do you need to join them?

Correct. Tools only match identical IDs, so you map one system to the other. Ensembl BioMart or the R package org.Hs.eg.db both do this.

Next in Track 1