Does Empirical Finance Replicate? AI answers the question.

Published empirical finance findings are usually reproducible in their original samples but lose much of their magnitude and significance in later data.

• The study rebuilds 1,328 results from papers published between 2000 and 2020 in the Journal of Finance, Journal of Financial Economics, and Review of Financial Studies; 1,005 have enough later data for out-of-sample testing.
• In sample, about 75% of results retain their published sign and statistical significance, with a median replicated coefficient equal to 93% of the published estimate; only 1.3% reverse sign and remain statistically significant.
• Out of sample, only 42% retain both sign and significance, or 48% after adjusting for shorter and less precise post-sample periods, while the median coefficient falls to 45% of the original estimate.
• Coefficients remain stable during the original sample and drop by about 0.28 when that sample ends, with no clear further decline after publication, a pattern more consistent with data snooping or sample selection than with investor learning.
• Persistence depends strongly on initial evidence: about 31% of the weakest-quartile results survive out of sample after precision adjustment, compared with approximately 72% of the strongest-quartile results; publication-selection models explain only about one-fifth of the observed decay.

Takeaway: Investors should treat published effect sizes as starting points rather than permanent constants, favoring findings that survive independent replication, multiple samples, and different market environments while remaining cautious even when the original evidence is statistically strong.