Hypothesis ledger: pre-registered hypotheses and their verdicts

State a hypothesis and the test that will answer it before you have seen any evidence, attach the runs that test it, and decide which hypotheses the evidence supports across all of them at once, with the false discovery rate controlled whatever the dependence between them.

What it doesCopy Anchor Link

Write the hypothesis in words, choose the test that will answer it and fix what that test must be run with: a recognition site and the null model it is judged under (and, if you like, the one record), for site avoidance; a gene and the genetic code it is read with, for codon outliers; a human gene and the trait it is claimed to affect, for gene–trait falsification. Choose the direction (fewer, more, or either way). Then run the test in its own panel, or for a gene–trait hypothesis from the hypothesis itself, and attach the run to the hypothesis.

Evidence is refused, with the reason beside the hypothesis, when the run used another test, changed a setting the hypothesis fixed, or comes from data the hypothesis already holds evidence from. The false discovery rate level and the calibration are fixed once any evidence exists. Nothing is dropped: a hypothesis with no evidence stays in the ledger and counts in the decision.

The ledger is part of the project and is there after a reload. Every pre-registration, every recorded and every refused piece of evidence, and every import and reset is a step in the project's Timeline. Export writes the whole ledger, every hypothesis and every trace, as one file; importing it replays every registration and every piece of evidence under the same rules and gives the same verdicts.

Gene–trait hypothesesCopy Anchor Link

A hypothesis that a human gene regulates a trait ("ITK regulates IL-2 production") is tested the way the POPPER agent tests one (Huang et al. 2025): by experiments that could falsify it, each run once, stopped as soon as the evidence is strong enough. You register the gene and how the trait shows itself in two public tables:

• GTEx expression. Either the trait's own gene or genes (IL2 for IL-2 production), and the test is how closely the gene's expression follows theirs across GTEx's 54 tissues; or the tissues the trait lives in, and the test is how much more the gene is expressed there than elsewhere.
• GWAS Catalog. The trait categories an involved gene's associations should fall in (immune system disorder, inflammatory measurement, and so on); the test counts the studies that map an association inside the gene in those categories.

Each test ranks the gene among all 19,107 protein-coding genes GTEx measures, so its p is exact without assuming any distribution. The GTEx test runs first; its e-value is multiplied by the GWAS test's only if it has not already passed 1/α, and a test after the stop is shown as not consulted. A symbol the tables do not know, a tissue or a category that does not exist, is refused before the hypothesis is registered, because a registration cannot be corrected later.

Beliefs and what to follow upCopy Anchor Link

Before any evidence is attached, you can say how strongly you believe each hypothesis (definitely true, maybe true, uncertain, maybe false, definitely false); once its evidence is in, say it again. Follow up next then ranks the hypotheses by how far the evidence moved your belief, so the result that surprised you most is the one to look at first. The verdicts never read a belief.

What to run next asks the other question: not which result was surprising, but which run that has not happened yet would teach the most. Give each candidate test the power you expect of it and, if you like, what it costs in whatever unit is scarce (days, reactions, sequencing lanes), and each is scored by the information its result would carry about the hypothesis it tests, in bits. A test of a belief already near certain teaches little however strong the test, and a strong test of a belief near even teaches most, which is why the ranking is not simply by power. Where the order by total information and the order by information per unit of cost disagree, both are shown and the disagreement is stated. A hypothesis you have not stated a belief about is left out and named, never ranked against an assumed even chance. The verdicts never read this ranking either.

A whole programme from the assistantCopy Anchor Link

The assistant can look at a corpus without running any test (each record's length, GC and file), propose hypotheses from that, and run them as one programme: every hypothesis is registered before any test runs, each test runs once over the corpus, the evidence is attached and refused by the same rules as here, and the decision is made across all of them. When that programme continued this ledger, its result opens here; otherwise it is saved to the project as a ledger file you can import. The same programme is a workflow step and an API call.

When you’d use thisCopy Anchor Link

• Keeping a search honest. Registering the test before the run is what separates a finding from a test chosen after seeing which one passed.
• Testing several hypotheses on the same data. The decision is valid under any dependence between them, so hypotheses about overlapping sites or related genomes can share a ledger.
• Handing on the record. The exported file carries what was asked, how it was tested, on which data and with which settings, which is what a reviewer needs to check a claim.

What this computesCopy Anchor Link

• p-value of each attached test, in the registered direction: the low tail for fewer, the high tail for more, and twice the smaller tail (at most 1) for either way, for site avoidance; the G-test's p for codon outliers
• e-value of each p by the Vovk–Wang calibrator κ·p^(κ−1), with κ your choice (0.5 by default)
• combined e-value of a hypothesis, the product of its e-values; 1 when it has none
• e-BH across every hypothesis at level α: the k strongest are supported when the k-th reaches K/(α·k), with K the number of hypotheses, so the strongest needs e ≥ K/α
• co-expression of a gene with the trait's genes: the Pearson correlation of log₂(TPM + 1) across the 54 GTEx tissues, averaged over the trait's genes
• tissue specificity of a gene: its mean log₂(TPM + 1) in the registered tissues minus its mean in the others
• GWAS study count of a gene: the GWAS Catalog studies with an association mapped inside it whose trait falls in a registered category (intergenic hits, which name two flanking genes, are not counted)
• rank p of each gene–trait test: the share of the 19,107 genes whose statistic is at least the gene's own, which is exact when the gene is exchangeable with the rest
• sequential stop: the gene–trait e-values multiply in the registered order and stop at the first product above 1/α
• Type-I rate of a gene–trait setting: with every gene in turn as the hypothesis, the share the setting would support, measured over the exact null rather than assumed
• verdict of each hypothesis: Supported, Not supported or No evidence
• refusals: evidence from another test, from another value of a fixed setting, or from data already counted, each with its reason and the records it covers
• belief before and after the evidence, each answer scored 1, 0.75, 0.5, 0.25 or 0 towards true and added to a Beta prior of (0.5, 0.5); the answer after the evidence counts twice
• Bayesian surprise of a hypothesis: the Kullback–Leibler divergence of the belief after from the belief before, which orders Follow up next; a surprise is a divergence of at least 5, and whether the belief crossed 0.5 (the published paper's condition) is reported beside it
• expected information gain of a candidate test, Lindley's measure: the mutual information between the hypothesis and the test's outcome, reported in bits and in nats, with the posterior each outcome would leave. For a hypothesis tested significant or not, it is exact from the belief, the power and the level α; for a count of successes out of n trials under a Beta belief, it is the exact sum over the n + 1 possible counts; for n readings of a quantity under a normal belief with normal noise, it is ½ log(1 + nσ₀²/σ²)
• information per unit of cost of a candidate test, its bits divided by the cost you gave it, and the smallest number of readings that would reach a target number of bits
• literature check of a hypothesis, against Europe PMC: papers whose title or abstract names the site (and its enzymes) or gene, the organism and the claim (avoided or enriched, codon usage), listed to read. A hypothesis is known only once you confirm a paper reports it; with no papers it is not found for that exact query and date, which bounds one search and never makes a finding new

Every result states, in words generated from the ledger, which hypotheses are supported and with what e-value (or how far the strongest falls short), which hypotheses have no evidence yet but still count, which supported hypotheses have no literature check recorded, and which combine evidence from several records, whose shared history means the product is pooled evidence rather than independent replication. For a gene–trait hypothesis it also says that its two tests are correlated across genes, gives the measured Type-I rate of its own settings, says when no study in the registered categories maps inside the gene, and says when the gene named is one of the trait's own genes, whose co-expression is 1 by construction.

ControlsCopy Anchor Link

Worked exampleCopy Anchor Link

The eleven genomes of the site-avoidance example, with three hypotheses registered before any run: that the 250 kb E. coli K-12 segment carries fewer KasI sites (GGCGCC) than its own 5-mers predict, that phage λ carries fewer SpeI sites (ACTAGT) than its own 5-mers predict, both under the maximal-order Markov null, and that every genome carries fewer AvrII sites (CCTAGG) than its base composition predicts.

One site-avoidance run under the maximal-order null, attached to each:

• H1 is supported: p 5.42 × 10⁻⁴⁴, e 2.15 × 10²¹.
• H2 is not supported: p 0.256, e 0.988. λ's SpeI rarity is already explained by its CTAG 4-mers.
• H3 is refused for all eleven records, because the run used the maximal-order null and H3 registered base composition. It stays in the ledger with no evidence, so the decision is over K = 3 and the strongest needs e ≥ 30.

Gene–trait: three genes from an IL-2 screenCopy Anchor Link

The Target validation starter registers three genes from the IL-2 benchmark of the POPPER paper, whose ground truth is a genome-wide CRISPR screen of IL-2 production in primary human T cells (Schmidt et al. 2022): ITK and MAP4K1 are among the screen's 50 strongest hits, PTPN22 is not. Each is tested by co-expression with IL2, then by GWAS studies of immune system disorders and inflammatory measurements, at α 0.1.

• ITK is supported on GTEx alone: r 0.733 with IL2, which only 11 genes reach (p 0.00063, e 20.0). The GWAS test is not consulted.
• MAP4K1 is not supported: it ranks 327th on co-expression (e 3.82) and no study in those categories maps inside it (p 1, e 0.5), a product of 1.91. The screen calls it real; these two tables cannot show it.
• PTPN22 is supported, e 91.6: it ranks only 340th on co-expression (e 3.75), but 24 studies map inside it, which only 7 genes reach (e 24.4). The screen found no effect on IL-2. Its GWAS signal is autoimmune disease, the trait registered, not the IL-2 the screen measured.

With every one of the 19,107 genes in turn as the hypothesis, these settings support 157, a Type-I rate of 0.8%. On the whole IL-2 benchmark they support 8 of the 50 true genes and 3 of the 49 false ones that can be placed; on the IFN-γ benchmark, 6 of 49 and 0 of 49. POPPER reports a power of 0.58 at a Type-I rate of 0.08 on IL-2: its agent designs many more experiments per hypothesis, and these two tests trade that power for a Type-I rate that is measured exactly.

LimitationsCopy Anchor Link

• Supported is not novel. A supported hypothesis is a finding of this data; whether it is already known is a separate check against the literature.
• A word match is not knowledge. Enzyme names collide with gene and protein names: the search for KasI avoidance in E. coli finds a paper on mutagenesis at the NarI sequence and one on the fatty-acid enzyme KAS I. That is why papers are listed to read rather than counted as proof.
• The product of a hypothesis's e-values is an e-value when each test is valid given the ones before it. Different records of related lineages share history, so evidence pooled across them is not independent replication.
• Evidence comes from the last run of Site Avoidance or Codon Outliers, or from a discovery programme run by the assistant or a workflow.
• A gene–trait test is evidence of association in public data, not of regulation. Co-expression and a GWAS signal are what a regulator would show, and so are genes downstream of the trait or sharing its cell type. The GWAS test answers for the registered categories, which can be broader than the trait (PTPN22 above).
• The two gene–trait tests rank the same gene, and across genes the ranks are correlated, so their product is not guaranteed to be an e-value; that is why each setting's Type-I rate is measured.
• Co-expression was added after the tissue test alone was measured on the IL-2 benchmark and found too weak (3 of 50 true genes supported); the IFN-γ benchmark was not used to choose it.
• GTEx v8 symbols are matched to current HGNC symbols; a gene whose symbol has been withdrawn, and the non-coding genes of the screens, cannot be tested.
• A belief and its surprise rank what to look at next; they say that a mind changed, which a wrong belief does as readily as a right one, and never enter the verdict.
• An expected gain is only as good as the power you claimed. It is what the test would teach if it has the power you gave it; a test that is weaker than claimed teaches less, and the ranking cannot know that. A shared data source is named rather than discounted, because how much evidence two tests of the same table really share depends on how each uses it.

Where the numbers come fromCopy Anchor Link

The calibrator and e-BH follow Vovk and Wang and Wang and Ramdas, and are checked against POPPER's own implementation and, through e-BH's identity with Benjamini–Hochberg, against R's p.adjust. The p-values are those of the attached tests, each checked against its own reference. The belief and surprise arithmetic is checked against AutoDiscovery's released code (Agarwal et al. 2025), whose defaults differ from its paper's (a flat prior, single weighting and the 0.5 crossing); the code's are used and the paper's crossing is reported. The gene–trait statistics, every gene's rank p and the verdict on every gene of POPPER's IL-2 and IFN-γ benchmarks are checked against R, computed independently from the same GTEx v8 and GWAS Catalog tables; the benchmark genes are drawn exactly as POPPER's own benchmark draws them, and our GTEx table agrees with the copy POPPER published on all but 253 of 971,514 shared values (genes whose symbol two Ensembl genes share). A discovery programme was run 60 times on each of three corpora where every hypothesis is false and supported nothing in any of them. The expected information gain is checked against R's entropy package on every table it can build, and its general estimator and closed forms against pyro's own optimal-experimental-design code.