GFFBase¶

A genomic-annotation engine built for whole-genome scale: a Rust GFF3/GTF parser, DuckDB columnar storage, and a zero-copy PyArrow interface — with the gffutils API preserved, so most code migrates by changing one import.

GFFBase reads GFF3 and GTF at whole-genome scale and answers the questions a genomics pipeline actually asks — spatial queries over intervals, hierarchy walks from gene to transcript to exon, and bulk extraction into Arrow, pandas or polars without a Python loop in the middle. The storage engine is DuckDB, so a database is one portable file and every query is columnar.

What you can do with GFFBase¶

🧬 Query intervals at genome scale

Ask for a region and get the features that overlap it, from an R-tree index built during ingest.

3. Spatial queries

🌳 Walk the annotation hierarchy

children, parents and the batched forms, backed by a transitive closure so depth costs nothing at query time.

2. Standard relational queries

🔄 Migrate from gffutils

The legacy API is preserved surface-for-surface. Most scripts move by changing one import.

Migrating from gffutils to gffbase

⚡ Extract in bulk, without a Python loop

children_batched(format="arrow") returns one Arrow table for thousands of anchors.

5. Vectorized ML API — the ones that make GFFBase fast

📊 Read the measured numbers

Head-to-head benchmarks on the canonical human annotations, with the machine, versions and commit of each run recorded.

Performance

🧰 Work from the command line

Build, inspect, validate and migrate a database without writing Python.

Command line

🧹 Read the files real sources publish

Byte-order marks, CDS-only and AUGUSTUS GTF, Liftoff ids, tar archives, truncated gzip: each read as it was meant, and reported.

Noisy and non-standard files

🎛️ Tune memory and threads

The defaults and why they were chosen, what ingest costs stage by stage, and when a loop should become one batched query.

Memory and performance tuning

Measured performance¶

Every figure below is generated from the committed benchmark artifact, never typed in. See Performance for the full sweep and Benchmark methodology for how it was run.

gffbase ingests every corpus faster, 1.92× to 3.62× against gffutils, into a smaller database. Batched extraction, spatial indexing and SQL over the whole corpus come on top. peak RSS is ingest plus exhaustive validation, not what the default path costs.

Corpus	Format	Lines	gffbase ingest	legacy ingest	speedup	peak RSS (ingest + full validation)	spatial qps	batched (5 k anchors)
GENCODE v49 (basic)	GTF	6,068,892	3 min 36 s	6 min 54 s	1.92×	53.47 GB	728 ±1% (n=5)	614 ms / 1.93 M desc
GENCODE v49 (basic)	GFF3	6,066,054	3 min 13 s	9 min 45 s	3.03×	61.93 GB	770 ±1% (n=5)	666 ms / 1.93 M desc
RefSeq GRCh38.p14	GFF3	4,932,571	2 min 5 s	6 min 31 s	3.13×	26.69 GB	588 ±1% (n=5)	443 ms / 999 k desc
CHESS 3.1.3	GFF3	2,761,061	37.0 s	2 min 14 s	3.62×	2.93 GB	678 ±3% (n=5)	155 ms / 161 k desc
MANE v1.5 (Ensembl)	GFF3	524,834	15.6 s	45.1 s	2.89×	4.00 GB	890 ±0% (n=5)	136 ms / 156 k desc

Install¶

pip install gffbase

Note

This site documents gffbase 0.3.0, released 2026-09-27: ingest 2.4-2.9x faster, in under a third of the memory on whole-genome files; prefetched loops; and noisy files read as they were meant (see the changelog). Upgrading from 0.2.0 fixes defects that silently lost data; upgrading from 0.1.0 also fixes two SQL injection vulnerabilities; see the security advisory.

Quick start¶

from gffbase import create_db

with create_db("gencode.v49.annotation.gtf.gz", "gencode.duckdb") as db:
    # GENCODE ids carry their version.
    for tx in db.children("ENSG00000139618.19", featuretype="transcript"):
        print(tx.id, tx.start, tx.end)

    for feature in db.region("chr17:43044295-43125483", featuretype="exon"):
        print(feature)

See Installation for the supported platforms, and Quickstart for a worked example that runs end to end.

Getting started

Using GFFBase

Background

Reference

Project