Phase 19 Hardening Decision
Phase 19 was a targeted hardening phase for the locked validated lane. Its purpose was not to expand the claim, but to test whether the locked closure result survives stronger uncertainty models, multi-fold temporal validation, an external regulator-side baseline, an alternative anchor, and a lead-time analysis.
Tracks and findings
- Institution-cluster bootstrap. 1,000 UNITID-level resamples on the locked test cohort. Joint-weak AUROC point estimate 0.864; cluster-bootstrap 95 % CI ≈ [0.806, 0.914]. Paired-difference vs the strongest single-variable baseline: mean +0.072, 95 % CI ≈ [+0.002, +0.138], about 98 % of bootstrap samples positive. The lock survives this hardening.
- Temporal multi-fold validation. Four sliding train/test time-block splits. Joint-weak AUROC 0.77–0.86 across all four folds; tied with simple low six-year graduation rate alone in Fold 1 (2015 test); A/G has a higher point estimate over the best single-variable baseline by +0.05 to +0.07 in Folds 2, 3, and 4. The signal is not specific to the locked 2017–2018 window.
- ED Financial Responsibility baseline (descriptive only). Federal composite-score data was fetched via the Urban Institute Education Data Portal API (a public mirror). Coverage stops at 2016, so the head-to-head test on the locked test cohort (2017–2018) cannot be run. On the descriptive train cohort 2014–2016, the federal composite produces AUROC ≈ 0.747; the simple low-graduation-rate baseline produces AUROC ≈ 0.788 on the same cohort. The federal composite is a real closure-relevant signal, but no head-to-head superiority claim against A/G is made.
- Heightened Cash Monitoring (HCM) anchor. Source-blocked in this run; documented as a candidate for future work. No HCM evidence is used to support the locked claim.
- Closure lead-time analysis. Among 108 closing FASB × Sector 2 institutions, joint-weak scores are elevated above the within-cohort median across the entire T-1 through T-5 horizon. At the locked top-decile cutoff, about 13 % of pre-closure rows are flagged at T-1 and about 25–27 % at T-4 and T-5. This is a multi-year early-warning pattern, not a coincident T-1 alarm; it is also not a perfect screen.
Integrated decision
The closure result holds. The classification is updated to “hardened FASB × Sector 2 private nonprofit four-year closure prediction” and stays inside the same validated lane. The hardening confirms the lock under appropriate clustered uncertainty and multi-fold temporal stability. It does not extend the validated lane.
What changed in this phase
- The reported confidence interval is now the institution-cluster bootstrap ≈ [0.806, 0.914]; the original analytic interval ≈ [0.801, 0.928] is preserved for compatibility.
- Multi-fold temporal evidence is added. The result is not specific to a single test window.
- The federal composite-score comparator is documented honestly as “runnable on a descriptive cohort, not on the locked test cohort.”
- HCM is documented as a future-work candidate.
- A multi-year lead-time pattern is documented, with explicit recall figures so the result is not over-claimed as a perfect screen.
What did not change
- The validated lane (FASB × Sector 2 private nonprofit four-year).
- The validated target (closure within five years).
- The forbidden-claim list, including the prohibition on universal higher-ed claims, on enrollment-collapse-prediction superiority, and on production-style framings.
- The decision to publish no per-institution scores and no institution-level rankings.