The Determinants of Income Segregation and Intergenerational Mobility: Using Test Scores to Measure Undermatching

Why it matters, in detail. Gap closure ≈ 0.31 + 32.8λ percent (need-affirmative) and 0.32 + 17.9λ (income-neutral), by construction of the mixture. Table IV’s λ̂ ≈ 0.85 is identified only under selection on observables with a score-sending proxy and modal-college treatment, the features Mountjoy–Hickman find under-correct. Their Texas estimates imply a portfolio-controlled share of about 0.2. By contrast, CDF’s waitlist and matriculation designs at the Ivy-Plus margin imply ≈0.76–0.78 for top-1% outcomes, and between ≈0 (waitlist, s.e. 1.70) and ≈0.96 (matriculation) for top-quartile outcomes. Transport adds a second issue: λ is identified on mean ranks but applied to threshold probabilities. With λ ∈ [0.4, 0.6], the need-affirmative reduction is 13.4–20.0% (income-neutral 7.5–11.0%), instead of 26.5% (14.6%).

Evidence. App. Table XI: need-affirmative gap closure = 0.31 + 32.76·λ (max residual 0.04 pp). λ = 0.6/0.5/0.4/0.3 gives 20.0/16.7/13.4/10.1% instead of 26.5%; income-neutral gives 11.0/9.3/7.5/5.7% instead of 14.6%. App. Table XII (heterogeneous effects) gives 21.3%. Table IV unrounded (workbook sheet ‘Table 7’): 1.0032, 0.9069, 0.9025, 0.8567, 0.8502, 0.85; the CI is [0.827, 0.874] for col 5 and [0.821, 0.879] for col 6. Mountjoy & Hickman, NBER WP 29276, §4.3.4: designs ‘relying solely on application controls (e.g. SAT/ACT score sending behavior) may not fully correct for selection, and thus may overestimate the relative value-added of more selective institutions’. M&H fn 50: Chetty et al. (2020)’s specification ‘only modestly attenuates outcome differences across colleges … possibly because their specification involves features we flag in Section 4.3.4 as liable to under-correct for selection bias, including defining the treatment as the modal rather than initial college, having application proxies only, and including students who only apply to the college they end up attending’. M&H Table 3: the value-added signal SD is 0.165 × the raw SD. Adjudicator λ-analog from M&H Online Appendix Table B.2 (earnings; 29 Texas publics vs UT-Austin, unweighted): the slope of admission-portfolio VA on ‘typical controls’ college effects is 0.21 (0.19 with portfolio FE only). Chetty, Deming & Friedman, NBER WP 31492 (rev. Aug 2025), Table 4, Ivy-Plus vs flagship, waitlist / matriculation / observational: predicted top-1% probability 5.01 (1.31) / 5.14 (0.39) / 6.59, a ratio ≈0.76–0.78; top 25% 0.01 (1.70) / 3.06 (0.82) / 3.19; mean rank 0.25 (0.91) / 1.78 (0.47) / 1.85. In §4.4 CDF find a strong observational-vs-portfolio relationship ‘with the exception of colleges in Texas’. Dale & Krueger (NBER WP 17159, Table 3): the school-SAT coefficient falls from 0.076 to −0.001 with self-revelation controls. Zimmerman (2014, JOLE Table 5): FIU admission raises quarterly earnings by $1,593 (s.e. 604), 22% of baseline, at one local margin. Black, Denning & Rothstein (NBER WP 26821): students gaining Texas flagship access gain, and displaced students show no declines in graduation or earnings.

Why this severity. The abstract’s mobility number is assumed up to the value of λ rather than estimated. Credible later evidence spans λ ≈ 0.2–1.0, which moves the headline between about 7% and 33%. A central skeptical value (0.4–0.6) cuts it by a quarter to a half. The direction survives for any λ > 0 and C1–C3 are unaffected, so the issue is major, not critical.

The authors' response. The abstract calls the assumption ‘strong’. App. Tables XI–XII report the full λ grid and a heterogeneous-effects scenario (21.3%). Footnote 34 attributes the divergence from Dale–Krueger to more precise parental-income controls. Later, two of the authors (Chetty, Deming & Friedman 2023, rev. 2025, §4.4 and fn 36) replicate Mountjoy–Hickman’s Texas null but argue that ‘Texas is unique’, and that portfolio-conditional and observational value-added are strongly related in California and for Ivy-Plus colleges.

Why it matters, in detail. C5 is presented as an empirical finding, but λ̂ ≈ 0.85 is identified only under selection on observables plus a score-sending proxy. The decline of λ̂ as controls are added is the usual signal that unobservables would push it lower. The comparison with Zimmerman’s RD covers one margin (FIU vs Miami Dade for academically marginal Florida students) and is unreported, so its implied CI cannot be checked. DK’s self-revelation results (0.076 → −0.001) are not straightforwardly ‘consistent’ with an 80% causal share of selectivity-related differences.

Evidence. NBER WP p.4; Table IV and notes; fn 34–35 (pp.19–20); pp.21–22; App. G and App. Table X (p.38). Dale & Krueger (2011) NBER WP 17159 Table 3. OI research summary (https://opportunityinsights.org/wp-content/uploads/2021/12/undermatch_summary.pdf, p.4). OI project page https://opportunityinsights.org/paper/undermatching/ (‘equalizing attendance rates for students with the same test scores would reduce the outcome gap by 15%. Going fully to the “all college goer” benchmark would reduce the outcome gap by 25%’). App. G also says CFSTY’s estimates (0.016, 0.012) lie ‘within one standard error’ of DK’s −0.001. That is false with the conventional s.e. the paper quotes (0.012: 1.42 and 1.08 s.e.), though true with the cluster-robust s.e. (0.018) reported in the same DK table.

Why this severity. No number changes. But the abstract’s ‘consistent with prior work’, the ‘at least 80%’ and a GPA control that does not exist overstate the empirical status of the single parameter that drives C4, and the key external check (Zimmerman) cannot be verified.

The authors' response. Footnote 34 acknowledges the difference from Dale & Krueger and gives two reasons: more precise parental-income controls, and SAT may predict earnings more strongly today. The abstract calls the assumption ‘strong’.

Why it matters, in detail. Hot-deck imputation on (q, state, tier, earnings) is consistent only if scores are missing at random given those variables. If non-takers are selected on latent ability, imputed scores are too high, and low-income non-takers are over-allocated to higher-score seats. Conditioning on tier biases toward too little reallocation, so the net sign is unknown. Because imputed scores are matched on own adult earnings, the imputed score and the outcome are linked within cells, which matters for the earnings draws. The de-meaned QQ evidence validates the shape of within-cell dispersion (about 15% too wide) but not levels.

Evidence. pp.10–11, fn 16–17; App. Fig. II notes. Adjudicator re-estimation from appx_paper_fig_6.dta: slope 1.148, constant 0.20, R² 0.9993; imputed/actual q95–q5 spread 1.14; largest ventile gap 38.6 points. Missing-score share by parent quintile implied by App. Table V and Table III: ≈46% (Q1) vs ≈13.5% (Q5). App. Table V (observed scores only): ≥1300 bottom-quintile share 3.1%, vs 3.7% with imputation.

Why this severity. The imputation covers a quarter of college-goers and nearly half of the low-income students whose reallocation drives C1 and C4. Its key assumption is untested at the level that matters: that non-takers resemble takers with the same earnings, tier, quintile and state. However, the observed-scores-only check shows C2 is robust, and no bias of known sign has been demonstrated.

The authors' response. Footnote 17 says that using SAT data only ‘yields very similar results’. App. Table V and an NPSAS check (App. Table VI) are offered as robustness checks, although the NPSAS check itself renorms income to the tax data and imputes scores.

Why it matters, in detail. A monotone shift is harmless within single-test states. In mixed states, however, it misranks ACT-only takers by about ⅓ SD relative to SAT-takers. That contaminates P(q | s, state, race) in each score cell, and hence both the income-neutral compositions and the donor cells of the earnings simulation. The national ≥1300 composition moves by only ≈0.1–0.3 pp, toward even fewer low-income high scorers. The sign of the effect on the ‘missing middle’ and on C1’s middle-class gains depends on the unobserved within-state income gradient of ACT-only test-taking.

Evidence. Summit Educational Group, ‘Concordance table: 2016 vs. 2018’ (Wayback 20220120073111), 2016 column: ACT 30 → 1390–1410, 28 → 1310–1340, 27 → 1280–1300, 21 → 1060–1090. Dorans, Lyu, Pommerich & Houston (1997), reprinted in College Board RN-07 (1999) Table 3 (recentered SAT I V+M): ACT 30 → 1340, 29 → 1300, 28 → 1260, 27 → 1220, 21 → 990. College Board ‘Concordance Tables’ (May 9, 2016) Table 10, old SAT (CR+M) → new SAT: 1000 → 1080, 1200 → 1270, 1300 → 1360, 1400 → 1450. Adjudicator script 10: inflation is +60 to +90 points for ACT 18–30 (up to ≈+105 at lower scores), ≈+78 averaged over N(21.8, 4.7); ≈44% of ACT-only students counted as ≥1300 would fall below 1300 on the old scale. If 10–20% of the ≥1300 group were such students (Q1 share ≈5%), the 3.7% bottom-quintile share would become ≈3.4–3.6%. Auditor A’s IPEDS-2001 old-scale approximation of App. Table IX-B (re-run) matches all six selective tiers within 0.1–2.4 pp.

Why this severity. This is an undisclosed, uncorrected error in building the key conditioning variable for a quarter of the sample. Its effect on national high-score shares is small and conservative for C2, and tier-level checks against IPEDS show no large distortion; but within-state reallocations in mixed-test states cannot be checked.

The authors' response. None located. Footnote 17 reports that running the imputation on SAT scores only ‘yields very similar results’; that bears on the imputation, not on the counterfactuals.

Why it matters, in detail. With fixed seats, fixed value-added and quintile thresholds recomputed with non-college-goers held fixed, rank gains for Q1–Q4 are mechanically matched by rank losses for Q5. The gap narrowing would be smaller if displacement from elite tiers is nearly costless for the displaced (as BDR find), or if value-added responds to composition through peer, network or resource effects. Holding Q1 gains fixed and halving Q5 losses gives (22.02 − 2.61 − 1.62)/22.02, i.e. a 19.2% narrowing. Drawing donors from inframarginal low-income enrollees likely overstates marginal entrants’ outcomes.

Evidence. p.13; p.20; App. F (p.37); Table III. Workbook sheet ‘Table 8’: P(top 20% | Q5) 40.24 → 38.29 → 37.01; P(top 20% | Q1) 18.23 → 19.49 → 20.83; share of college-goers in the top quintile 29.54 / 29.49 / 29.47% (adjudicator script 02). Black, Denning & Rothstein, NBER WP 26821 (AEJ: Applied 2023). App. Table XII: with zero causal effects for within-selective moves of quintiles 2–5, the narrowing is 21.3%.

Why this severity. The structure is disclosed as a benchmark and partly explored (App. Table XII). But the headline’s size depends on simulated losses for displaced high-income students that the best quasi-experimental evidence does not find. Halving those losses alone would cut the need-affirmative narrowing from 26.5% to about 19%.

The authors' response. p.13: the counterfactuals ‘should not be thought of as policy proposals, but rather as benchmarks’. p.20 acknowledges the peer-effect assumption, and App. Table XII reports a heterogeneous-effects scenario.

Why it matters, in detail. H&A’s predictor is a multivariate block-group/ZIP/high-school regression trained on verified CSS Profile incomes and calibrated to an 8% false-positive rate among non-bottom-quartile students, not a tract mean. The within-area income–score correlation that CFSTY document can still bias any area-based predictor for high scorers, and the point that ACS thresholds differ from tax-based incomes is valid. So the qualitative reconciliation may hold even though the description does not, but the 17% vs 5.0% and 39% vs 16.6% gaps are never decomposed.

Evidence. NBER WP p.3 fn 5; p.12 and fn 21; App. C (pp.34–36) and App. Table II (2006: P25 $23,250 vs $27,161; median $50,000 vs $58,791). Hoxby & Avery (2013), BPEA Spring, pp.11–13 and p.15 fn 15.

Why this severity. This affects only the explanation of a disagreement with prior work. CFSTY’s own high-scorer composition estimates rest on administrative income data and are unaffected.

Why it matters, in detail. The bonus profile is a calibration target, not an estimated policy response. The comparison with admission-stage preferences for small groups is an order-of-magnitude analogy, not evidence of feasibility or of how applications and matriculation would respond. ‘Eliminated’ and ‘fully desegregates’ hold only approximately under state × race constraints (≈81% of the exposure gap closed).

Evidence. Abstract; pp.3, 13, 16–17, 22; Table III Panel C; App. Table VIII Panel C; paper_fig_6a.dta (22.81 → 29.29 vs a 30.79 benchmark). Espenshade, Chung & Walling (2004) p.1431; Arcidiacono, Kinsler & Ransom (2019) Table 10 (admits-weighted retention 29.6%, ratio 3.38; paper 29.4%, 3.4).

Why this severity. This is interpretive overstatement in the abstract and conclusion. The numbers themselves are correct, and elsewhere the paper says the scenarios are benchmarks.

The authors' response. p.17 notes that the counterfactual could alternatively be implemented by raising application or matriculation rates; p.13 calls the scenarios benchmarks.

Why it matters, in detail. All discrepancies are at most 1.6 pp on derived ratios (closure shares) or are presentational. The like-for-like exposure-gap closure using the authors’ own figure data is 62.6%, not 63.9%. The 0.3% residual at λ = 0 is a constant offset across columns (intercept ≈0.31 in every column), consistent with threshold recomputation.

Evidence. (i) ‘38%’ selective-tier closure (pp.2, 15): the unrounded value is 39.55%, and Table III’s own rows (31.3% → 18.9%) imply 39.6%; 38% comes from the rounded 7.3/8.6/10.7. (ii) ‘63.9%’ exposure-gap closure (p.14) uses an actual exposure of 22.5% (also App. Table VIII), but the authors’ figure input paper_fig_6a.dta gives 22.81% (and 38.87% vs ‘38.8’ for Q5), implying 62.6%. (iii) Income-neutral top-1% gap ‘2.3 pp’ (p.20) vs 2.38 unrounded (Table V cells 3.1 − 0.7 = 2.4). (iv) The App. Table XI λ = 0 row shows 0.3% in every column, while p.21 says the impact ‘mechanically decreases to 0%’. (v) App. Table IV notes say ‘3.6%’, while the table says 3.7% (figure data 3.68%). (vi) ‘Approximately 18.5% of children have $0’ earnings (p.10) vs 18.3% implied by Table I (81.68% employed). (vii) The OI project page says Ivy-Plus closes ‘just 6% of the gap’ (the paper says 9%; unrounded 8.90%) and that ‘7.3% of low-income students currently attend Ivy-plus schools’, omitting ‘with an SAT of 1400’. (viii) App. G’s ‘within one standard error’ (see F2).

Why this severity. Rounding-level slips of at most 1.6 pp on secondary ratios; no conclusion is affected.

Why it matters, in detail. None of these errors appears in the computations. The bonus is added to lower-quintile scores as the formula and the Table III notes say, and the conditioning in the earnings draws is stated correctly on p.20. The errors impede reading and replication rather than change results.

Evidence. Conclusion (p.22) states the 80% assumption as ‘conditional on parental income, race, and state’, but earnings draws condition on test scores, not state (fn 30). Footnote 25 (also QJE fn 45) reverses the equivalence: a bottom-quintile s-scorer is treated like a top-quintile (s+160)-scorer, not the reverse. Footnote 2 cites a nonexistent ‘Online Appendix L’ (Appendix H is meant). Footnote 5 points to ‘Section V.A’, but the discussion is in Section III. App. H cites ‘Appendix Figure V’ for the SAT–earnings plot, which is Figure VI. p.2 says ‘Barron’s Tier 6 or higher’ for the paper’s top six tiers (Barron’s 1–5). λ and θ are mixed (pp.21–22; App. Table XI notes). Table II notes describe trend statistics that are not in the table. Figure I notes omit state from the imputation cell. App. F omits race from earnings-donor cells and the state × race allocation constraint. Parent income is described as measured at ‘aged 15–19’ although fn 14 uses calendar years 1996–2000. App. D and App. Fig. I notes describe the 1978-cohort college definition differently. App. G says ‘30’ colleges where the sample has 31.

Why this severity. Documentation slips. The formulas, the Table III/V notes and the body text state the intended methods, and the outputs are consistent with them.

Why it matters, in detail. The simulation sample is not documented. Its overall margins are close to, but not identical with, the no-imputation sample, so ‘actual’ benchmarks differ across tables by up to 3 pp for small for-profit tiers and by about 0.1 pp overall.

Evidence. Table I and App. Table III notes. Workbook sheet ‘Table 6’, row ‘All colleges’ (0.106595 / 0.307897). Public Online Data Table 2 recomputations (adjudicator script 04; Auditor A script 02, re-run). Tables II and III.

Why this severity. Restricting the sample is sensible for the test-score linkage, and the excluded students are mostly at small and for-profit colleges. The problem is the lack of disclosure, not the numbers.

Why it matters, in detail. The 15% and 25% rank-correlation reductions, the value-added gap statistics and the alternative-definition robustness cannot be verified from public material. The quintile transition matrices do not identify within-quintile rank covariances.

Evidence. Replication README; code/draw_paper_figs_tabs.do (exhibit-only); NBER WP pp.7, 9, 20–21, fn 15, 35; App. F; workbook sheet ‘Table 7’ L7:L10.

Why this severity. Confidentiality legitimately limits what can be released, and the tabulated top-quintile-gap analogue (26.5%) corroborates the untabulated headline. The gap mainly prevents verification.

The authors' response. Footnote 15: confidentiality rules prevent releasing statistics that use test-score or race data by college.

Why it matters, in detail. This is an upper-bound calculation because the gradients are not conditioned on SAT. It is a small upward bias in the simulated gap narrowing, concentrated in the top-1% metric.

Evidence. Equation (1) (quintic in parent rank) vs p.20 and App. F (donor cells by quintile). Public Online Data Table 6 gradients; Auditor A script 09, re-run by the adjudicator: understatement of counterfactual Q5 rates ≈0.075–0.095 pp (top 20%) and ≈0.05–0.07 pp (top 1%).

Why this severity. The quantified effect is ≈1.5% of the 5.8 pp narrowing for the top-20% outcome and ≈5–8% of the 0.93 pp narrowing for the top-1% outcome.

Why it matters, in detail. The conditional rate is overstated by a factor of 1.64 because high-top-1%-share colleges get weight from their Q1 students but contribute their whole-student-body rate. The row is internally inconsistent with Table II columns 8/10 and Table V Panel A.

Evidence. NBER WP App. Table III; QJE Online Appendix Table III (same values). Replication code app_paper_tab_3.do line 124, read as text: ‘egen kp99_cond_parq1_mean = wtmean(k_top1pc), weight(count * par_q1)’. Adjudicator script 09: correct conditional 0.61% (no-imputation sample) and 0.56% (with imputation); the unconditional-based formula reproduces the published 1.00% and 0.92%. Column 1 (all children, 0.22%) is correct. Found independently in the companion audit of the QJE version.

Why this severity. This is a descriptive appendix table. It overstates the top-1% success rate of low-income college-goers by about 65%, but it does not feed the counterfactuals, which use microdata.

Why it matters, in detail. The population effect ≈ Σ_q share_college(q)·ΔP(top 20% | q, college), with non-college-goers held fixed and the college-goer top-quintile share essentially unchanged (29.54 → 29.47%). It gives a 12% narrowing of a 28.1 pp gap under need-affirmative allocation (≈7–8% if λ = 0.5) and 7% under income-neutral allocation.

Evidence. Abstract; pp.4, 20–22. Adjudicator script 05: college attendance by parent quintile 28.7/39.6/51.2/64.7/81.2%; population P(top 20%) by parent quintile 8.65/12.72/17.71/24.20/36.72%; need-affirmative changes among college-goers (+2.61 pp Q1, −3.23 pp Q5) translate into +0.75 pp and −2.62 pp in the population.

Why this severity. The abstract and the definition are correctly scoped, so readers of the abstract are not misled. The unqualified statements in the text and conclusion invite a population-wide reading that is about twice too large.