How Elastic Are Preferences for Redistribution? Evidence from Randomized Survey Experiments

Why it matters, in detail. The reported coefficients are reduced-form intent-to-treat effects of a multi-component negative prime: -0.14 to -0.16 points on 0-4 scales, or 12-13% of the political gap. Reading them as trust effects requires the exclusion restriction. The prime also lowered perceived deservingness of the rich (-5.6 pp) and shifted vote intention. The first stage is significant at only one threshold of the 4-point scale (F about 7.5-8), and the Wald ratios exceed twice the liberal-conservative gap. The share of the omnibus policy response mediated by trust is unidentified. The minimum-wage versus transfer contrast offered as mechanism evidence is statistically indistinguishable from zero.

Evidence. The adjudicator's re-estimates (scripts/02_trust.py, 09_extra_tests.py) reproduce Tables 8-9 exactly. The prime moved other attitudes: 'rich deserve their income' -5.55 pp (p=0.020; online App. Table 15). That is a first-stage outcome in Table 4 but is omitted from Table 8. Inequality 'very serious' rose +5.47 pp (p=0.079) and Democratic 2014 vote intention fell -4.62 pp (p=0.074). First stage: binary trust -5.82 pp (p=0.004), but the 4-point trust scale moves only -0.048 (p=0.25); only the 'most of the time' cut moves (cumulative thresholds +0.007, -0.058, +0.003). The implied Wald ratios are 2.39 (aid), 2.63 (food stamps) and 2.81 (housing) scale points per unit of the trust dummy, 2.0-2.3 times the entire liberal-conservative gap. In the control group, trust is negatively (and insignificantly) associated with transfer support (-0.16 to -0.21). No individual Table 9 outcome survives a within-table Holm correction at 5% (smallest adjusted p about 0.06); a three-transfer index does (-0.152, p=0.0076). Min-wage minus transfer effect, in political-gap units: +0.11 (se 0.11, p=0.30) as published and +0.04 (p=0.51) with corrected coding (F2). Peyton (2020) re-analyses the KNSS data (APSR supplement S5.7): first-stage F=7.48 and full-scale trust effect -0.04 (p=0.36). In three experiments that raised and lowered trust, he finds 'substantively negligible' effects on support for redistribution.

Why this severity. The paper presents the causal effect of trust as its most novel result and as part of the abstract's explanation for small policy effects. With a bundled prime, an unvalidated and empirically contradicted exclusion restriction, a fragile first stage and implausible implied magnitudes, that causal claim is not supported. The reduced-form effect of the bundle on transfer support survives, and C1-C3 are unaffected.

The authors' response. The paper hedges its mechanism analysis: 'we do not view our attempts to tease out the mechanisms behind the main results as definitive' (p.1481), and 'one should not view our analysis ... as definitive or exhaustive' (p.1499). It argues that the marginal rise in inequality concern would, if anything, mask a negative trust effect (p.1500). Neither auditor located a published reply to Peyton (2020).

Why it matters, in detail. The pooled -2.9 pp averages a -6.3 pp effect in one round with effects of about zero elsewhere (round 4 vs the rest: -0.053, p about 0.03). It loses significance under reweighting, under dropping round 4 and under Lee trimming. The text overstates its robustness, and the discarded worst-case bounds bracket zero.

Evidence. From scripts/01_omnibus.py and 09_extra_tests.py. Trust by round: +0.001 (p=0.98), -0.000 (p=0.99), -0.039 (p=0.13), -0.063 (p=0.002). Without round 4: -0.012 (p=0.39). CBS-reweighted (32 cells): -0.016 (HC1 p=0.23). Lee bounds [-0.052, +0.085]; range-only bounds [-0.125, +0.155]. In the pooled sample the 4-point trust scale does move (-0.072, p=0.001). 2013 NBER WP App. Table 2: trust L -0.0387, U +0.0710. Both auditors report the same leave-one-out and reweighting results.

Why this severity. This weakens the descriptive half of the trust explanation (that information lowers trust), not C1-C3. The effect is real in the cleanest round, the one with the lowest differential attrition, so it is fragile rather than refuted.

The authors' response. The paper reports App. Tables 11-12 and states that the trust result is 'robust and at times stronger for the low-differential-attrition round, the round where we expect our identification assumptions to be most robust' (p.1494).

Why it matters, in detail. Evidence the authors held but did not report reverses the sign of the claimed inequality-to-trust effect: +4.9 pp in the 2014 arms and -2.4 pp (n.s.) in 2012. It places the omnibus trust decline with the growth/tax slide. It also shows that the 2014 inequality-only treatment moved 'very serious' concern by only 1.7-4.6 pp, against 10.4 pp for the full omnibus package.

Evidence. From scripts/04_followups.py and 05_dropped_waves.py. Slider comprehension answers are present for 98-100% of inequality-arm respondents and 0% of controls. 2014 inequality arm, waves 1, 2 and 5 (602 treated, 593 controls): trust +4.94 pp (se 2.06, p=0.017), wave 5 alone +7.32 pp (p=0.006). Inequality 'very serious' +4.57 pp (p=0.078; wave 5 +1.69 pp, p=0.60); 'inequality increased' +6.87 pp (p=0.002); top rate +0.48 and millionaire tax +0.6 pp, both n.s. In wave 5 the arm also raised support for aid (+0.115, p=0.08) and food stamps (+0.150, p=0.03). 2013 NBER WP App. Table 7 (2012 round 4), effects on trust: pure inequality -0.024 (n.s.), pure growth -0.038 (p<0.1), both -0.067 (p<0.01).

Why this severity. The arms contradict the paper's reading of the trust decline and show smaller concern effects for inequality information alone in 2014. But they corroborate C2, do not touch C3, and the trust claim also rests on other evidence (F1, F3).

Why it matters, in detail. The top-rate effect's 95% upper limit of 2.0 pp (about 20% of the gap) is an observed-sample statistic. Under monotone selection the identified set reaches +5.5 pp (54% of the gap). The paper's imputation exercise assumes that, within ideology groups, attriters in both arms have the same outcomes, which is the very assumption in question.

Evidence. From scripts/01_omnibus.py and 09_extra_tests.py (within-round, weighted). Lee bounds: concern [0.060, 0.209]; estate [0.328, 0.462]; top rate [-1.72, +5.48] pp (-17% to +54% of the 10.2-pp gap); millionaire tax [-0.059, +0.092]; minimum wage [-0.069, +0.078]; petition [0.033, 0.175]. Range-only bounds: estate [0.176, 0.469]; concern [-0.033, 0.247]. App. Table 10 reproduces: C column 0.0666; L column 0.0980 only with the contaminated mean, 0.1033 when cleaned. Auditor B obtained identical Lee bounds; Auditor A's pooled versions are similar.

Why this severity. C1 and C3 survive the bounds; C3 survives even worst-case bounds. The policy effects are still small in the observed sample, but their claimed precision does not hold under selective attrition.

The authors' response. The paper reports the differential attrition and shows covariate balance among completers (Table 3). It reports results for the low-differential-attrition round and for follow-up experiments with negligible differential attrition (App. Tables 3, 11-12).

Why it matters, in detail. The abstract's causal attribution conflates information with an advocacy frame. The information-only effect is +10.9 pp (+52%), about 38% of the framed estate-only effect and 31% of the omnibus effect. Demand effects cannot be ruled out for the framed versions; the neutral effect is the cleaner estimate of correcting misinformation.

Evidence. From scripts/04_followups.py (Table 7 reproduced). Control 'increase' share 21.0%. Emotional +28.9 pp (to 49.9%, x2.38); neutral +10.9 pp (to 31.9%, x1.52); difference 18.0 pp (se 2.6). Neutral information mainly lowers support for cutting the tax: 'decrease' falls from 39.4% to 20.8% and 'left as is' rises from 39.6% to 47.8%. Omnibus: 17.1% to 52.8% (x3.09; Table 5). Sides (2011 working paper, pp.16-17): correct information about who pays raises estate-tax support by a marginal effect of 0.16, comparable to pro-tax value arguments.

Why this severity. The magnitude attributable to information alone is about half the headline 'doubling' (+52% vs +100-200%). But the estate tax remains the clear exception (neutral effect 0.41 of the political gap vs about 0.1 for income-tax outcomes), and the paper discloses the neutral result.

The authors' response. The body discloses it: 'a purely clinical information treatment with no emotional component increases support by 50 percent instead of more than doubling it' (p.1491). The conclusion says 'even a four-sentence description providing information on its incidence significantly increases support for the policy' (p.1505).

Why it matters, in detail. A difference in significance is not a significant difference. In the common-control wave, the empathy and policy-link treatments have statistically indistinguishable effects on transfer programs (differences of -0.003 to +0.09 scale points). The experiment therefore does not establish that linking concern to programs is what moves transfer preferences.

Evidence. Wave-5 joint model (scripts/04_followups.py), policy minus poverty: aid -0.003 (p=0.96); food stamps, consecutive coding, +0.043 (p=0.53); housing +0.089 (p=0.16); minimum wage +0.142 (p=0.010), or +0.276 with the published coding. The empathy arm itself raises aid +0.133 (p=0.04) and food +0.179 (p=0.01). The unreported inequality arm raises aid +0.115 (p=0.08) and food +0.150 (p=0.03). All 388 wave-5 controls appear in both Table 10 and Table 11. Both auditors obtained the same pattern with slightly different specifications.

Why this severity. This narrows the 'disconnect' mechanism to the minimum wage. The policy treatment's effects relative to control are real, and C1-C3 are untouched.

Why it matters, in detail. Typical respondents' rank is overstated by roughly 10-15 percentiles. Evidence from Cruces, Perez-Truglia and Tetaz (2013) and Karadja, Mollerstrom and Seim (2017, REStat 99(2)) suggests that learning one is richer than believed lowers support for redistribution. Any bias would therefore attenuate the policy effects the paper calls inelastic. The counterfactual-growth slider uses the same unit concept.

Evidence. Slider: 39% of households below $25,000 and 79% below $73,000; comprehension answer key P50 $33,800, P90 $108,000, P99 $386,000. Census P60-239 (2010): household median $49,445 and P90 $138,923 (Table A-3); 25.7% below $25,000 and 68.2% below $75,000 (Table A-2). Piketty-Saez TabFig2010 Table 0: tax-unit P90 $108,024, P99 $352,055 (2011prel $366,623; 2012prel $393,941) (scripts/07_slider_thresholds.py). Auditor A's household P90 of about $154,000 (interpolated) overstates; Census publishes $138,923.

Why this severity. This is an undisclosed inaccuracy in the customized information at the heart of the omnibus treatment. Its effect on the estimates cannot be identified, and it cannot plausibly reverse C1-C3.

Why it matters, in detail. Including the dropped waves raises the pooled top-rate effect to about 1.1 pp (about 11% of the gap) and the millionaire-tax effect slightly. The AER's sample description and footnote 20 are inconsistent with the posted data and the design history. The heterogeneous treatment content was a legitimate reason for exclusion, and it should have been stated.

Evidence. 2013 NBER WP pp.6-7 and App. Tables 4 and 9 give wave dates and content. Its Tables 4-5 report N=4,477-4,521, top rate 1.131/1.096 (p<0.05), concern 0.113/0.115 and estate 0.360/0.357. Dropped waves with omnibus-style controls (scripts/05_dropped_waves.py; N about 794): concern +0.166 (p<0.001); trust -0.034 (n.s.); top rate +1.97 pp (p=0.010; 29% of the gap); millionaire tax +7.0 pp (p=0.013). Wave 1 alone: top rate +4.37 (p=0.002). In the posted AER data every round is about 50/50 (401/410, 577/585, 616/617, 682/681). The adjudicator's re-estimate of SES heterogeneity in the AER sample is directionally similar but insignificant (degree holders, food stamps +3.0 pp, p=0.13; interaction p=0.17).

Why this severity. Headline estimates barely change across versions (top rate 1.10 to 0.93; estate 0.357 in both), so the conclusions survive. But the excluded data, whose point estimates exceed the AER's stated 20%-of-gap upper bound, were dropped without disclosure, a material transparency caveat.

Why it matters, in detail. Under the corrected ordering, the policy treatment's minimum-wage coefficient is 0.18, not 0.32. In gap-scaled terms it is no larger than the food-stamp effect (0.143 vs 0.149). The effect works on intensity (slightly to significantly increase), not on the share favouring an increase. The trust prime's minimum-wage effect is -0.07 (0.095 of the gap), close to its transfer effects; neither difference is significant. Raw food/aid/housing coefficients fall 25-31%, but their gap-scaled effects and significance survive.

Evidence. Raw-code profiles (scripts/03_scale_codes.py), from the policy/poverty survey, wave 5. Minimum-wage raw code 1: mean policyview 3.74, Obama-2012 share 0.80, millionaire-tax support 0.91. Code 4: 3.15/0.64/0.75. Code 2: 2.50/0.32/0.39. Code 3: 2.89/0.47/0.26. Code 5: 1.75/0.11/0.06. The ordering 1 > 4 > 2 holds in every one of the seven 2014 batches (three trust-survey batches, the estate survey and the inequality pilot and wave 5). The rare codes 3 and 5 (1-2% of answers) are the least supportive wherever their cells are not tiny. Re-estimates use the same samples and controls (scripts/04_followups.py). Policy treatment (Table 11) minimum wage: 0.323 (0.31 of the political gap) becomes 0.180 (0.14; p=0.0008), against food stamps 0.229 (0.15) and housing 0.176 (0.13). The share wanting any increase moves +2.9 pp (n.s.); the share wanting a significant increase moves +10.4 pp (p=0.001). Trust prime (Table 9) minimum wage: -0.004 becomes -0.070 (0.095 of the gap, p=0.16), against transfer effects of 0.12-0.13 of the gap. Poverty treatment (Table 10): 0.047 becomes 0.020. With consecutive coding, food stamps fall from 0.313 to 0.229 (Table 11) and from 0.177 to 0.133 (Table 10); App. Table 14 emotional housing falls from 0.236 to 0.163. For these items gap-scaled effects and significance are essentially unchanged.

Why this severity. C1-C3 are untouched. But the coding inflates the policy treatment's minimum-wage effect by about 80%; its gap-scaled effect more than halves under the corrected order. It also removes the evidence for two claims built on the minimum wage: that it shows 'the largest effect' (p.1504; Conclusion p.1505), and that distrust leaves it unaffected (p.1501).

Why it matters, in detail. An IV-style ratio of the policy reduced form to the concern first stage is about 9 pp of top rate per unit of 'very serious' (0.93/0.104). That exceeds the covariate-adjusted cross-sectional slope (5.4 pp), the opposite of what a disconnect implies. Because the omnibus also moved other beliefs, the ratio is an upper bound on concern's own effect. The cross-sectional slope is not causal, so this is a benchmark, not a refutation.

Evidence. From scripts/01_omnibus.py, section J. The first stage is +0.104 on 'very serious' and +0.306 on the 1-5 concern scale. Predicted effect (control-group slope x first stage, with the 1-5 scale version in parentheses) vs observed treatment effect: top rate 0.56 (0.82) vs 0.93; millionaire tax 0.016 (0.032) vs 0.050; minimum wage 0.018 (0.031) vs 0.033; food stamps 0.014 (0.025) vs 0.015; EITC 0.013 vs 0.021; estate tax 0.020 vs 0.357. Auditor A (I16): concern moves +0.23 SD, millionaire tax +0.115 SD (19% of 'room to move' vs 15% for concern), top rate +0.05 SD.

Why this severity. C1 and C2 stand as descriptive facts. What goes is the evidence for treating the small omnibus policy effects as anomalous, which is the premise of the mechanism claims (C4-C5).

The authors' response. The paper notes that dividing by the political gap 'is hardly a perfect metric' and reports raw effects (p.1489).

Why it matters, in detail. Adjusted retention is 58% (unadjusted 79%), with a 14% follow-up response rate. The decay is not significant, so 'persistent but imprecisely measured' is the defensible summary.

Evidence. From scripts/08_misc_checks.py. First survey 0.3367 [0.0953]; follow-up 0.1951 [0.0910], p=0.0347. Paired change -0.142 (se 0.112, p=0.21). Unadjusted effects 0.245 and then 0.194 (p=0.0085). Panel controls rise from 0.128 to 0.179; treated stay at 0.373. The printed 'First' control mean of 0.180 is the all-respondent mean. The follow-up file contains only data round 7.0; residual df=93. See also 2013 NBER WP Table 9.

Why this severity. Persistence of a positive estate effect is supported: there is no significant decay, and treated respondents' support is unchanged. Only the summary wording, one significance star and some labels are wrong.

Why it matters, in detail. The stated incidence is 0.10%, against 0.18% for taxable estates or 0.38-0.44% for filers. The qualitative contrast with respondents' beliefs (most think a majority pay) is unchanged.

Evidence. IRS SOI 11es02yd.xlsx: 4,415 taxable returns (9,447 filed) for 2011 decedents. 13es01yd.xlsx: 4,699 taxable (11,309 filed) for 2013 decedents. NCHS deaths: 2,515,458 (2011) and 2,596,993 (2013), giving 0.176% and 0.181% (scripts/06_source_checks.py). Auditor A's filing-year ratios give 1.49-1.99 per 1,000.

Why this severity. The order of magnitude and the correct multiple-choice answer ('less than 1%') are unaffected. No effect on the estimated treatment effects can be inferred.

Why it matters, in detail. Measured by real GDP per capita (1948-80 vs 1980-2010, the longest span available from FRED), the high-tax era's growth premium is about 0.5 percentage points a year, not the slide's 2.4. The slide is advocacy rather than neutral information. That matters for reading the trust results, because the 2012 decomposition places the trust decline in arms containing this slide.

Evidence. Piketty-Saez TabFig2012prel Table A0: average income per tax unit, excluding capital gains, $43,452 (1980) to $51,737 (2010), or 0.58% a year; tax units 99.6m to 156.2m. Real GDP per capita (FRED A939RX0Q048SBEA): 1.79% a year in 1980-2010 vs 2.28% in 1948-80 (scripts/10_growth_measure.py). Auditor A's peak-to-peak rates: 1913-29 +0.50%, 1929-80 +1.87%, 1980-2007 +1.05%.

Why this severity. The paper calls the slide a 'raw correlation' and says its treatments erred toward persuasion. A more persuasive than accurate slide, if anything, strengthens the finding that income-tax preferences barely moved.

The authors' response. The paper presents the figure as 'at least as a raw correlation' and says the treatments 'erred on the side of presenting information we thought would indeed move those preferences' (pp.1484-1485).

Why it matters, in detail. Content inaccuracy in a persuasion treatment. It adds to the bundling problem (F1) but has no separately quantifiable effect.

Evidence. Appendix A.5 item 2; Appendix Figure 8 (viewed by the adjudicator). FEC Citizens United case summary and TI CPI 2013 (Auditor B, results/source_checks.json): US score 73, 15 OECD members higher and 18 lower.

Why this severity. The reduced-form effect of the actual prime is still identified, and the prime is avowedly negative. The inaccuracy matters only if the prime is described as factual information, which F1 already covers.

Why it matters, in detail. Appendix robustness tables are computed on mis-specified samples and denominators. The main pooled estimates include all rounds and are unaffected.

Evidence. In scripts/01_omnibus.py, rounds 1-2 reproduce the published 'High' column (top rate 2.003, estate 0.365, trust 0.001); rounds 1-3 give 1.440, 0.325 and -0.012. Section G gives L 0.0980 vs 0.1033 when cleaned. In 08_misc_checks.py, the item denominators include 679 control rows, and the Table 3 F-tests give p=0.066 and 0.113. See ineq_Omnibus.do, attrition-study block.

Why this severity. None of these changes a main-text estimate. The corrections mostly favour the authors' conclusions or leave them unchanged; the exception is that the trust estimate for the 'High' group becomes negative but insignificant.

Why it matters, in detail. The liberal skew of mTurk relative to CBS is overstated by about 70% (0.59 vs 0.35 on the 1-3 scale). The employment gap is about 1.6 pp, not 12 pp.

Evidence. From scripts/08_misc_checks.py: political views 1.586 as printed vs 1.829 excluding 'don't know' (mTurk 2.176); age 48.99 vs 48.10; the 'Other' value 0.0209 is the Asian share (true 'other' 0.026). Auditor A: mTurk employment is 0.571 including the self-employed, vs 0.465 printed.

Why this severity. This is a descriptive comparison only. The direction of the mTurk sample's skew (younger, more liberal, more educated) is right, and the treatment effects are unaffected.

Why it matters, in detail. Presentation errors with no systematic favourable pattern.

Evidence. Adjudicator reproductions (scripts/01, 02, 04, 08), Auditor A (I14) and Auditor B (I11). The December 2014 NBER revision contains the same slips.

Why this severity. Each error is recoverable from the tables, several understate significance, and none changes a conclusion.

Why it matters, in detail. The quote overstates baseline support for repeal among all respondents, since respondents without an opinion are excluded from Slemrod's base.

Evidence. Slemrod, December 2003 Brookings working-paper version of the NTJ 2006 article, p.8 and footnote 11 (cached text read by the adjudicator). The 49/31/20 belief shares are quoted correctly.

Why this severity. A background citation that does not bear on the experimental estimates.