Data integrity: survivorship, point-in-time inputs, and missing data

A dataset can be tidy and still know the future

Data integrity means the historical dataset represents what existed, what was observable, and what was tradable at each decision time. Removing nulls and standardizing columns is only part of the job.

Three errors are especially damaging:

• Survivorship bias removes instruments, funds, or strategies that failed or disappeared.
• Point-in-time error gives a historical row information before it was released or replaces the original value with a later revision.
• Missing-data error silently converts “unknown” or “not available” into a convenient observation.

All three can improve a backtest without changing the visible strategy. The model simply receives a better past than a real researcher had.

Survivorship changes the opportunity set

Illustrative example. Suppose an equity strategy is tested on the companies in today’s index from 2005 onward. The dataset includes firms that survived long enough to remain in the index and excludes many that were removed, acquired, delisted, or failed.

The strategy is no longer answering “what could an investor have selected in 2005?” It is answering “how would today’s survivors have looked in 2005?” That changes returns, risk, sector composition, and the frequency of severe losses.

A point-in-time universe records membership, identifiers, listings, corporate actions, and delisting outcomes as they changed. It also defines whether a security was liquid and eligible before the decision, rather than using its later success to infer eligibility.

A value has more than one date

For an accounting or economic observation, distinguish:

1. the period the value describes;
2. the public release timestamp;
3. the effective time when the strategy may use it;
4. later amendment or revision timestamps.

A fiscal-year revenue figure may describe December 31 but be filed in February. Assigning it to December gives the strategy information weeks early. Replacing the original filing with a later restatement gives every historical decision the clean final value.

Point-in-time research retains publication history or uses a reproducible lag that is conservative for the source. The lag is not a magic cure: different issuers and datasets publish on different schedules, so source timestamps are better when available.

Missing does not mean zero

A missing value can mean:

• the instrument did not exist;
• the market was closed;
• the field was not reported yet;
• the source failed;
• the observation is structurally undefined;
• the value is known but absent from this vendor.

Replacing all of those with zero gives them one invented meaning. Dropping the row can select healthier instruments. Forward-filling can carry stale information through an event where it ceased to be valid.

A credible policy records the reason when possible, then declares what the strategy does: exclude the instrument, delay the decision, use an age limit, impute under a justified model, or stop with a data-quality failure.

The treatment follows the reason:

The same blank cell can therefore lead to exclusion, delay, an explicit stale value, or a failed run. A universal fill rule conceals those different economic states.

Adjustments can move information backward

Corporate actions and continuous futures require transformations. Split adjustments keep price history comparable; dividend treatment distinguishes price and total returns; futures rolls combine separate contracts.

The transformation must match the research question. A back-adjusted futures series is useful for returns and charts but may not represent a historical tradable price level. An adjusted equity close can be correct for return calculation while being inappropriate for a literal limit-price rule unless the rest of the series and order logic are adjusted consistently.

A concrete point-in-time example

Illustrative example. Assume a strategy ranks companies on an earnings measure for a portfolio formed at the end of March.

• Company A filed on February 15.
• Company B filed on April 10.
• Company C later restated its February filing in June.

At the March cutoff, A’s original filing is available. B’s new value is not. C’s original filing is available, but the June restatement is not. A dataset using all three final values contains future information even though each row has the correct fiscal period.

The research policy must decide whether B uses its previous filing, remains ineligible, or is handled another declared way. That choice affects the portfolio and must be preserved.

Integrity checks before strategy logic

These checks belong before parameter search. An optimizer can exploit a data artifact more efficiently than a human can notice it.

Common integrity mistakes

• Rebuilding past universes from current constituents.
• Using the fiscal period end as the publication date.
• Overwriting historical releases with final revised values.
• Treating missing observations as zero or harmless.
• Dropping failed instruments without their economic outcome.
• Joining by ticker without handling identifier changes.
• Mixing adjusted and unadjusted fields inside one rule.
• Ignoring a vendor or methodology change in the middle of the sample.

Further reading

• Shumway, “The Delisting Bias in CRSP Data” (1997) — Documents missing CRSP delisting returns and shows why excluding or mishandling failed securities biases historical stock-return data. It is a direct equity-backtest example of survivor and missing-outcome bias.
• U.S. Securities and Exchange Commission, “Search Filings” — Provides the authoritative EDGAR search, latest-filings, and submissions access points. Filing and amendment timing help distinguish when information became public from the period that information describes.
• Croushore and Stark, “A Real-Time Data Set for Macroeconomists” (2001) — Builds historical vintages of macroeconomic data and demonstrates how using final revised values instead of what was known at the time can change forecast evaluation.