Niels Johannesen, Gabriel Zucman · American Economic Journal: Economic Policy 6(1): 65-91

The End of Bank Secrecy? An Evaluation of the G20 Tax Haven Crackdown

16 problems found: 2 major · 5 moderate · 9 minor

What the paper claims

Bilateral treaty effect: an information-exchange treaty between a country and a haven lowers that country's non-bank deposits in the haven by about 11% relative to other country-haven pairs; if evaders own about half of deposits this is a ~22% fall in evaders' deposits, so only a minority of evaders (weighted by assets) moved funds.
The crackdown did not trigger significant repatriation: aggregate deposits in havens stayed roughly unchanged and OECD countries that signed more treaties did not see their haven deposits fall ('leaving roughly unchanged the total amount of wealth managed offshore').
'Key result': evaders shifted deposits to havens not covered by a treaty with their home country, so the crackdown relocated deposits to the benefit of the least compliant havens (those signing few treaties and outside the EU Savings Directive).
Deposits held through sham corporations (haven-registered deposits) may have responded strongly to treaties.
No compliance effect of treaties among EU holders of Swiss accounts; their compliance stays low (~10-20%, an upper bound); OECD's EUR 14bn of recovered tax implies only ~$350bn disclosed, against ~$6tn offshore.

Every problem found

1majorconfirmed · previously known · methodology

The paper's main story is that evaders moved money from havens that signed treaties to havens that did not. That pattern largely disappears when the authors' own model is made stricter (a version printed in their drafts but cut from the published table). It is also absent in the eight havens whose data are now public. It may hold for a few small havens, but it is not firmly demonstrated.

Technical detail and evidence

Why it matters, in detail. The coverage x (1-Signed) coefficient is identified mostly from haven-level growth differences: small havens that signed few treaties (Panama +82%, Cyprus +60%, Malaysia +44%, Macao +30%) grew fast. Once haven-year and saver-year effects absorb that, the saver-specific component that bilateral shifting predicts is 0.0040 (SE ~0.0046), and the gap between non-treaty and treaty havens (0.0040 vs 0.0030) vanishes. In public data for 8 havens the term is -0.0009 (CI excludes 0.012), and the deposit-weighted Figure 5 slope is -0.010 (p=0.48). Direction: the published specifications overstate relocation. The 1.2%-per-treaty inflow (1.8% for non-STD havens) is not a robust estimate, and dollar-level relocation is not shown.

Evidence. Replication workbook JohannesenZucman2014ExcelData.xlsx sheet 'Table4' cols [9]-[10] (also Table 4 cols 7-8 of the 31 Jan 2012 WP and cols [9]-[10] of the 23 Jan 2013 manuscript): coverage x (1-Signed) 0.0115 (p=0.0151) -> 0.0040 (p=0.3838), implied SE ~0.0046, 95% CI [-0.005, +0.013]; coverage x Signed 0.0022 -> 0.0030; Signed -0.1407 -> -0.0984 (p=0.22). Public BIS data, adjudicator's own panel and FE code (final/data/regress_public_output.txt): T1 col 2 analogue -0.1118 (p=0.050); T2 col 2 coverage x (1-Signed) -0.0009 (SE 0.0044; 95% CI [-0.0094, +0.0076]) vs published +0.0120; T2 col 1 coverage -0.0023 vs +0.0059; T2 col 3 (1-STD)(1-S) +0.0072 (p=0.20) vs +0.0183; OECD + saver-year FE +0.0052 (p=0.36); EU savers -0.0056, non-EU +0.0024; leave-one-haven-out -0.0063 to +0.0021. Auditor A: -0.0009; Auditor B: -0.0007. Figure 5 (DataFig5; fig5_sens_output.txt): baseline -0.0384 (SE 0.0139, p=0.015) reproduces; weighted by 2007 deposits -0.0104 (p=0.48); without Panama and Cyprus -0.0218 (p=0.22); every single-haven deletion stays significant (-0.030 to -0.044). Cayman (+$60bn) and Singapore (+$51bn) signed 18 and 19 compliant treaties, like Jersey (18) and Guernsey (19). Jersey 2007 -> 2011H1 (public BIS, jersey_decomp_output.txt): -$111.4bn, of which -$63.5bn vis-a-vis Swiss and -$20.3bn vis-a-vis UK non-banks; only -$32.1bn vis-a-vis non-haven countries. Menkhoff & Miethe (2019), DIW DP 1711, Sec. 4.4 and Fig. 7.

Why this severity. The authors call shifting their key result. It disappears in the specification that best isolates the bilateral mechanism and in the 74% of the sample whose data are public, and the cross-haven pattern needs small havens to be weighted equally with large ones. The direction may still hold for a few small non-cooperative havens, so the finding is weakened to 'suggestive' rather than reversed.

The authors' response. p. 85: 'When we include both country-year dummies and haven-year dummies, we still find a modest effect of treaties on deposits but are unable to identify a deposit shifting effect (results not reported).' No later reply located.

2majorconfirmed · newly identified here · overclaiming

The paper argues that money was not brought home because the total held in havens stayed about the same. But the amount that moved because of the treaties was only a few percent of that total, far too small to show up in such noisy numbers. The country-by-country comparison is too imprecise to tell either way, so the data cannot say whether evaders brought money home or moved it elsewhere.

Technical detail and evidence

Why it matters, in detail. Detectable effect versus noise: the outflow implied by Fig. 3 is $47-72bn (<=2.7% of the $2.68tn stock), against annual aggregate movements of -8% to +26%. The non-haven benchmark implies a 5-16% relative decline, and the authors' own FX-adjusted flows are -4.9%. Fig. 6's CI [-0.045, 0.032] includes the full-repatriation prediction (-0.006 to -0.016 per treaty). Because of time fixed effects, the regressions identify only differences between pairs. Direction: the paper overstates the evidence against repatriation. The evidence is uninformative rather than contrary.

Evidence. DataFig2-3 (fig3_public_output.txt): compliant-treaty group shortfall vs the no-treaty trend $46.6bn (2007 base) to $71.9bn (2008 base), i.e. 1.7%-2.7% of 2011 haven deposits ($2,677bn); $135bn (5.1%) with a 2006 base. Year-on-year changes in the haven aggregate 2005-2011: +6.8, +15.0, +25.6, +10.9, -7.9, -1.5, +1.6%. Havens vs non-havens: 2007->2011 -5.7% (~$163bn), 2008->2011 -5.0%, 2009->2011 -15.7% (~$500bn); haven/non-haven ratio 0.672 (2004) -> 0.569 (2011). DataFig5 'Cumulated exchange rate adjusted flows 2008q1-2011q2': total -$140.1bn = -4.9% of end-2007 stocks. Figure 6 (Data-Fig6): b = -0.0062, SE 0.0189, 95% CI [-0.045, +0.032] (row-shift corrected: -0.0069). Expected slope under full repatriation ~ (e^beta - 1) x w = -0.109 x (1/18 to 0.15) = -0.006 to -0.016 per treaty.

Why this severity. Half of the abstract's headline ('rather than repatriating funds') rests on evidence that cannot tell repatriation from relocation. Combined with F1, 'no repatriation' is not established, although nothing presented shows large repatriation either.

The authors' response. p. 73: the non-haven series 'might be an imperfect counterfactual ... but we can at least exclude that the G20 crackdown was followed by a significant drop in aggregate deposits in tax havens' (this narrower statement is correct).

3moderateconfirmed · newly identified here · internal inconsistency

Taken literally, the paper's own numbers say that once money-shifting is allowed for, deposits in treaty havens fell by only about 5%, a fall not clearly different from zero. Meanwhile other havens gained more than was lost. The paper's two headline numbers cannot both be right as stated.

Technical detail and evidence

Why it matters, in detail. A SUTVA violation inherent in the paper's own shifting result: with contaminated controls, the Table 1 beta (-0.1156) mixes the treaty-haven outflow (-0.05, insignificant, in col 2) with the forgone inflow (0.012N). The literal implied net change in a saver's total haven deposits is +3% to +12% for low coverage and -3% to -4% for high coverage. Direction: C1 as an outflow is overstated if C3 holds, and the 'lower bound for evaders' claim (p. 78) fails. These are unweighted log effects, so the dollar mapping is illustrative.

Evidence. Published Table 2 col 2: Signed -0.0498 (p=0.43), coverage x Signed +0.0001, coverage x (1-Signed) +0.0120. Adjudicator (adding_up_check_output.txt): treaty havens -4.8% (N=6) / -4.7% (N=15); no-treaty havens +7.5% / +19.7%; net total with covered share w=1/3: +3.4% / +11.6%; w=0.5: +1.3% / +7.5%; w=0.92: -3.8% / -2.8%. fn 15: -0.0498 + 6(0.0001-0.0120) = -0.1212 (correct arithmetic). Auditor A's adding_up.py gives the same.

Why this severity. No sign changes, but the paper's two headline numbers cannot both be taken at face value. Under its own model the size of the treaty-haven outflow is uncertain, and the 'roughly unchanged total' is not what the coefficients say.

The authors' response. fn 15: 'This coefficient is comparable to the coefficient found in column 2 of Table 1.'

4moderatelikely · newly identified here · methodology

The '11 percent' is an average over country-haven pairs in which a tiny pair counts as much as a huge one. Weighting pairs by the money they hold, the effect in the public data is about zero. It is concentrated in Jersey, the Isle of Man and Luxembourg and in treaties signed before the G20 push, and is absent in Switzerland. The effect on the money actually held offshore is therefore less certain than the headline suggests.

Technical detail and evidence

Why it matters, in detail. beta is an equal-weighted mean of pair-level log changes. The deposit-weighted analogue is ~0 (CI about +/-0.13), and haven-specific effects range from +0.25 (BE) and -0.004 (CH) to -0.23 (JE) and -0.77 (CL). Precision comes mainly from 2007-08 Crown-Dependency TIEAs (ATT(3-6) = -0.51); the 2009 cohort's ATT(3-6) is -0.12 (SE 0.09). Direction: the dollar-weighted response is probably smaller, and the evader response is less certain than (e^beta - 1)/s = -22% suggests. The finding does not overturn the existence of a typical-pair decline.

Evidence. Public BIS data, adjudicator (regress_public_output.txt, by_haven_output.txt, c1_robustness_output.txt, cohort_es_output.txt): weighted by initial (2003q4-2004q3) pair deposits beta = +0.0012 (SE 0.065; N=19,630) vs unweighted -0.104 on the same sample. Separate-haven estimates: CH -0.004 (SE 0.099), AT +0.097, BE +0.245, JE -0.232, IM -0.198, LU -0.158, GG -0.104, CL -0.767 (same as Auditor A). Cohort vs never-treated ATT over event times 0-6: 2007 -0.48 (6 pairs), 2008 -0.38 (15 pairs), 2009 -0.095 (SE 0.088, 87 pairs), 2010 +0.05, 2011 +0.15. Event times 3-6: 2008 cohort -0.505 (p<0.001) vs 2009 cohort -0.117 (SE 0.094, p=0.22). Observation-weighted clean ATT -0.087 (SE ~0.056) vs TWFE -0.112. 2007-08 cohorts: IM, GG and JE with DK, FI, IS, NO, SE, FO, NL, DE, GB. Paper pp. 75, 79.

Why this severity. The sign and typical-pair size of C1 are corroborated, including by Menkhoff & Miethe (2019) and O'Reilly et al. (2019). What becomes uncertain is the response of the money actually held offshore and its attribution to the G20-wave treaties. The '22% of evaders' deposits (weighted by assets)' reading is not what the estimator measures.

5moderatelikely · newly identified here · methodology

The claim that money went to the 'least compliant' havens rests on a number that mostly reflects non-European savers in all havens, Switzerland included. The non-cooperating havens the paper names (Singapore, Hong Kong, the Bahamas and Bahrain) are not even in the data used for that test.

Technical detail and evidence

Why it matters, in detail. Identification of 'non-Directive havens attract EU evaders' comes only from EU savers in CL, MO, MY and PA, about $46bn of deposits in 2011H1. The rest of the 0.0183 cell is non-EU savers in all 13 havens, for whom shifting in the public havens is +0.0024 (p=0.70). Direction: the estimate overstates evidence that relocation favoured 'least compliant' havens. The true magnitude is unknown.

Evidence. Do-file Part 6: STD_country = CH, LU, JE, GG, IM, KY, BE, AT, CY; STD = 1 if STD_country & EU15/EU10 & date>180 (EU2 date>186). Published p. 80 text and Table 2 col 3. Public BIS data (adjudicator = Auditor A): non-EU savers coverage x (1-S) +0.0024 (p=0.70); EU savers -0.0056 (p=0.41); col-3 analogue (1-STD)(1-S) +0.0072 (p=0.20), STD x (1-S) -0.0128 (p=0.01).

Why this severity. This undermines the 'least compliant havens' qualifier in the abstract and conclusion. It does not by itself refute shifting, which is assessed in F1.

6moderatelikely · newly identified here · methodology

The evidence that money held through shell companies responded strongly comes from only about a dozen havens, but the statistics treat it as if it came from hundreds of independent observations. Done properly on the available data, the result is no longer statistically reliable.

Technical detail and evidence

Why it matters, in detail. A Moulton / few-clusters problem: with treatment at the haven-quarter level and 13 havens, pair-clustered SEs understate uncertainty. In the public analogue, proper inference moves p from 0.001 to ~0.14 and the count measure flips sign. Direction: overstated precision. Fn 18's arithmetic (F13) understates the point estimate, but the inferential problem dominates.

Evidence. Adjudicator (table5_public_output.txt): share coefficient -0.305: pair-cluster p=0.001; banking-haven cluster p=0.143; wild cluster bootstrap-t (Webb weights, 8 clusters, B=1,999, null imposed) p=0.140. Count coefficient +0.0052 (pair p=0.013; wild bootstrap p=0.24) vs paper -0.0067. Auditor A: -0.31 / p=0.12 / p=0.12; count +0.0041. Published Table 5 p. 86; p. 87 caveat.

Why this severity. C4 is a secondary, cautiously worded result ('might have responded strongly'). Its statistical support is probably much weaker than reported, but the paper does not rest its main claims on it.

The authors' response. p. 87: 'the results in Table 5 rely on variation at the haven level rather than variation at the country-haven-pair level.'

7moderatelikely · newly identified here · methodology

To measure honesty, the paper counts how many Swiss account holders let their bank report them to their home tax office. But people could also pay the Swiss withholding tax and still declare the income at home, so the measure can miss honest behaviour. The paper's own numbers for 2011 are also somewhat higher than the '10-20 percent' it reports.

Technical detail and evidence

Why it matters, in detail. The measured statistic is D/(D+W) (notified over notified-plus-withheld interest). Compliance on the covered base is (D + cW)/(D+W), where c is the share of withheld interest also declared at home: with D/(D+W) = 0.2, c = 0.25 gives 0.40. The shell exclusion pushes the other way, so neither the level nor the bound is identified, and the Table A4 null concerns notifications only. Direction ambiguous. The measure also feeds Zucman's (2015) global 80%-undeclared assumption.

Evidence. ESTV 'Zinsbesteuerungsabkommen / Statistik 01.01.2009-31.12.2011' (/mnt/zdata/downloads/estv-euzins-jz2014/): Art. 1/8 shares and Art. 2 notifications with gross interest; France 2011: 55,603,617.89 CHF share, 33,158,260.71 CHF notified, which matches Data-Fig7 (Auditor B checked 291 cells, A checked all inputs). Data-Fig7 2011: declared 713.1m vs taxed 1,989.9m CHF -> 26.4%; unweighted mean 21.7% (treaty 22.8%, no-treaty 20.3%); 2006 weighted 19.4%. HMRC 'EUSD: exchange of information and withholding tax' (2005), Sec. 5: investors who have tax withheld 'may claim credit for the tax withheld'. Zucman (2015) online appendix, 'Fraction of offshore wealth which is not declared on tax returns'. Table A4 null closely reproduced by B (-0.0255 vs -0.0253).

Why this severity. This affects the interpretation of a secondary result: both the level of compliance and the meaning of the null. The treaty/no-treaty null on the notification channel itself reproduces.

The authors' response. fn 22: the figures 'are upper bounds' because the denominator excludes interest earned through sham corporations; pp. 88-89: the evidence is 'far from systematic'.

8minorconfirmed · newly identified here · overclaiming

The checks the authors used to show their comparison is fair are too imprecise to rule much out. Extra checks we ran mostly support the basic treaty effect, but they do not support treating the money-shifting pattern as proven cause and effect.

Technical detail and evidence

Why it matters, in detail. Power: the placebo's 95% CI (+/-0.19) contains beta = -0.116, so it cannot discriminate. The probit's zero-growth coding and treated-at-risk design attenuate the test. The pooled leads are flat, but the pre-2009 lead (-0.17) and the 2006-07 aggregate divergence (-14.8%) show non-trivial pre-period movements. Net: C1's causal reading is broadly supported; the causal reading of shifting is not tested by these designs.

Evidence. Table 4 col 1: -0.0248 (p=0.7963) -> SE ~0.096, 95% CI [-0.213, +0.164]; public reproduction -0.008 (SE 0.112). Do-file Table 3 block: 'replace depgrowth_l4to0=0 if dep==0 & L4.dep==0'. DataFig2-3 ratios 2006 0.580, 2007 0.494, 2008 0.519, 2011 0.449. Public event study (adjudicator, ref. t=-1): leads -8..-2 between +0.000 and +0.055 (all p>0.27); lags +0.04, -0.03, -0.06, -0.13 (p=0.02), 4+ -0.18 (p=0.002). Pre-2009 lead4 -0.169 (p=0.021; Auditor B's diagnostic, replicated). Entry-into-force coding -0.161 (p=0.048).

Why this severity. The bilateral effect passes the stronger tests run here. The problem is mainly the claim that the correlations 'can thus be considered causal' (p. 83), applied also to shifting, and the weak graphical evidence.

The authors' response. Sec. III.C: the two tests 'establish that we have a reasonably valid natural experiment'.

9minorconfirmed · newly identified here · selective presentation

The table showing that the money-shifting result disappears under a stricter test was in earlier drafts, but it was cut from the published article, which mentions the test in only one sentence.

Technical detail and evidence

Why it matters, in detail. No numerical effect. It is a presentation choice that moved the most demanding test of shifting (coverage x (1-S) = 0.0040, p=0.38) out of the table while the abstract and introduction elevated shifting to the key result.

Evidence. mirror-paper.txt (31 Jan 2012 WP) Table 4 cols 7-8; aeJ13-version.txt (23 Jan 2013) Table 4 cols [9]-[10]: Signed -0.1163 (p=0.0744) / -0.0984 (p=0.2175), coverage x (1-S) 0.0040 (p=0.3838), STD -0.3727. Published Table 4 (p. 84) note: 'Columns 3-10 consider the deposits held by nonbank agents'. 2012 abstract: 'caused a modest relocation of deposits between havens but no significant repatriation'; published p. 67: 'our key result'.

Why this severity. The result is disclosed in the text and in the public workbook, so readers are not misled about its existence. But removing it lowers the visibility of the main robustness failure of the paper's key result.

The authors' response. p. 85 sentence acknowledging that shifting cannot be identified with both sets of dummies.

10minorconfirmed · previously known · speculative assumption

The paper's statement that evaders cut their deposits by 22% assumes they own half of all haven deposits, a figure that comes from an assumption in earlier work rather than a measurement. Its broader conclusion, that most evaders did not move, holds unless evaders own less than about a fifth of deposits.

Technical detail and evidence

Why it matters, in detail. The evader response scales as (e^beta - 1)/s. With s anchored on an assumed deposit share of offshore wealth, '22%' could be 15-44% for s between 0.75 and 0.25. The qualitative conclusion is robust for s > ~0.22.

Evidence. Zucman (2013) QJE p. 1343: 'I assume ... that 25% of it takes the form of deposits and 75% of securities, as is the case in Switzerland ... $4.5 trillion (securities) plus $1.4 trillion (deposits)'. BIS haven deposits 2008 $2,904bn (DataFig2-3) -> 48%. (e^-0.1156 - 1)/s: s=0.5 -> -21.8%, s=0.25 -> -43.7%.

Why this severity. The paper itself shows that its qualitative 'minority of evaders' conclusion holds for s > 20-25%. Only the 22% figure depends on the assumption.

11minorconfirmed · newly identified here · data error

A coding slip meant that the check meant to remove currency swings actually exaggerated them for Swiss-franc and yen deposits. It affects only a side check, and probably not by much.

Technical detail and evidence

Why it matters, in detail. A mis-specified FX deflator for at most 4-9% of deposits contaminates the 'exchange-rate adjusted' estimates (Table 4 cols 5-6). The sign of the bias on beta is indeterminate: CHF-heavy Swiss treaty pairs were treated in 2009-10, before strong CHF appreciation. The headline unadjusted estimate is unaffected.

Evidence. Exchange rates.txt: CHF 1.248 (2003q4) and 0.8344 (2011q2) per USD; EUR 1.2496 USD per EUR. Factor applied at 2011q2: CHF 1.496 (correct 0.669); JPY 1.327 (0.754); MYR 1.258 (0.795); CLP 1.272 (0.786). Affected currencies are at most 4-9% of deposits (CHF plus 'other' currencies in Online Appendix Table A3), more in Swiss pairs. Published adjusted beta -0.0890 (p=0.095).

Why this severity. This affects a robustness check only. The authors' main argument (similar currency composition across groups, Table A3) does not depend on it, and the affected currencies are a small share of deposits.

12minorconfirmed · newly identified here · arithmetic or transcription

A small calculation error made the paper's shell-company example smaller than it should have been (45% instead of about 57%). Correcting it actually strengthens the authors' point.

Technical detail and evidence

Why it matters, in detail. The illustrative magnitude is understated by 1.3 points in aggregate (4.5% vs 5.7%) and by 12.7 points (45% vs 57%) owner-specific, a relative error of +28%. The error runs against the authors.

Evidence. Table 5 col 3: -0.5900. (exp(-0.59)-1) x 0.10 = -0.0446; exp(-0.059)-1 = -0.0573; /0.10 -> -57.3% vs -44.6% (adding_up_check_output.txt; Auditor B calculations.json).

Why this severity. Auditor B graded this moderate. It is graded minor here because the correction strengthens rather than weakens the authors' illustrative claim, which is itself conditional on an unobserved 10% ownership share and undermined more by F6.

13minorconfirmed · newly identified here · arithmetic or transcription

Ten countries in one chart were plotted with the wrong numbers of treaties because of a spreadsheet slip. Fixing it does not change the chart's message.

Technical detail and evidence

Why it matters, in detail. Transcription error in the x-variable for 10/34 observations; corrected b = -0.0069 vs -0.0062. No change in inference (see F2 on the plot's lack of power).

Evidence. fig6_counts_output.txt: e.g. Poland plotted with 11 compliant treaties (correct 1) and 35 treaties with havens (Norway's); Mexico 2 (10); Netherlands 10 (13); New Zealand 13 (8); Norway 8 (11); Portugal 1 (7); Slovakia 7 (1). Slope -0.0062 (SE 0.0189) -> -0.0069 (SE 0.0189). Confirms Auditor A's fig6_rowshift.py.

Why this severity. Purely presentational: the slope barely changes and the regressions do not use these counts.

14minorconfirmed · newly identified here · data error

A control for an EU tax rule starts one quarter too early in the code. It makes no practical difference.

Technical detail and evidence

Why it matters, in detail. A one-quarter-early indicator for a control. The coefficients of interest move in the 4th decimal (public data).

Evidence. Do-file lines 1663-1665 ('date>180', 'date>186'); Stata %tq 180 = 2005q1. HMRC (2005): the Directive 'takes effect from 1 July 2005'. Public data (std_timing_output.txt): T2 col 2 coverage x (1-S) -0.0009 -> -0.0006; STD -0.2175 -> -0.2143 (Auditor B: +0.0003 change).

Why this severity. A control-variable timing slip with demonstrated negligible impact.

15minorconfirmed · newly identified here · documentation gap

There are several small slips in labels, counts and wording. None of them changes the results.

Technical detail and evidence

Why it matters, in detail. No estimate changes. The only one with a testable consequence, the saver-side haven list, moves beta from -0.112 to -0.107 in public data (Auditor A).

Evidence. Published pp. 69-72, 77, 79, 85; Online Appendix D p. 9 and Table A1; workbook DataFig1, DataFig2-3 (haven-haven 568.4 / 2,677.3 $bn), DataFig5, Data-Fig6; do-file Part 6 OFC list and template.txt (47 hosts); reconstructed events (domlaw counts by host); GAO-11-730 (read via WebFetch): '894 U.S.-initiated outgoing requests closed during this period', Table 1 new outgoing requests 1,022.

Why this severity. Labelling, wording and documentation slips with no effect on conclusions.

16minorlikely · newly identified here · selective presentation

The paper's go-to example, France and Switzerland, does not actually show the effect: French deposits in Swiss banks rose after their 2009 treaty.

Technical detail and evidence

Why it matters, in detail. The motivating pair is a non-compliant/unreviewed treaty whose class shows no average effect, and Switzerland's own effect is ~0 (SE 0.10). This misleads only about where the average effect comes from (see F3).

Evidence. Reconstructed treaty events: CH-FR signed 2009-08-27, standard missing (unreviewed). Workbook Table1 unreported column: Signed x Compliant -0.1455 (p=0.027), Signed x (1-Compliant) -0.0183 (p=0.82). Public BIS: FR->CH non-bank deposits $15.77bn (2009q2) -> $17.69bn (2011q2); Auditor A: cumulative FX-adjusted flows 2009q3-2011q2 +$0.2bn vs -$16.2bn for all non-haven savers in CH. CH-only beta -0.004 (SE 0.099).

Why this severity. Presentational. 'Say' signals an illustration, but readers are likely to take the example as typical.

Claims examined and rejected

Problems one auditor raised that did not survive checking.

The '1,025 events' (p. 70) and the '1,090 treaties' (Online Appendix D) are inconsistent. — Both are correct: 1,025 counts events in 2004q1-2011q2, 1,090 counts all dated events. The adjudicator's comparison of the treaty reconstructions confirms both totals (Auditor A itself noted 'both are right').
Figure 5's slope falls to about zero when the domestic-law events the paper calls 'legally equivalent' are counted (Cyprus +36, Belgium +71). — Overstated. The collapse is driven by Belgium's 14 April 2011 law (66 non-haven events in the window's final quarter, too late to affect 2011H1 averages) and by Cyprus's non-compliant partners. Counting Cyprus's 10 compliant domestic-law partners leaves the slope at -0.037 (p=0.049). The paper's own Table 1 col 4 (no response to domestic-law events) justifies excluding them. The other Figure 5 sensitivities (deposit weighting; dropping Panama and Cyprus) are retained in F1.
The inverted exchange-rate adjustment likely makes the adjusted beta more negative than a correct adjustment would. — The direction is not established. CHF deposits are concentrated in Swiss pairs, many treated in 2009-10 before the strong CHF appreciation, which pushes the bias the other way. The coding error itself is confirmed (F12).
The published shifting estimate 'must' come from the five havens without public bilateral data, contradicting footnote 14. — Softened rather than accepted as stated. Given the close match of data vintages and of the Table 1 replication this is likely but not certain (specification or vintage differences cannot be excluded). Footnote 14 concerns the use of unobserved havens; the substantive problem is heterogeneity, which F1 records with confidence 'likely' for this inference.
Figure 3's 'no-treaty' group mislabels pairs with pre-2008 treaties (e.g. US-Cayman). — Not an error: the figure legend and note define the group explicitly as pairs that did not sign a compliant treaty between 2008:I and 2011:II.
Do-file comments disagree with the coded dates for LU-CH and AT-SG. — Immaterial: the dates fall in the same quarter, so no variable or estimate is affected.
The independent treaty reconstruction yields 1,018 rather than 1,025 in-window events (854 vs 861 new treaties), an unresolved discrepancy. — Resolved as an artefact of Auditor B's port. Its step counts deviate from those documented in the do-file comments (e.g. step 1: 514 vs 444; step 8: 195 vs 271; step 11: 522 vs 530). Auditor A's port matches those counts (except one documented pair) and the published 1,025/861/164 and 586/112/163 totals exactly. The 7 missing events (6 haven-haven pairs plus CH-SK) are in A's steps 8-9. Auditor B had correctly declined to attribute the gap to the authors.
Bermuda is explicitly excluded by footnote 8, so its treatment raises no inconsistency. — Bermuda is in Table A1's 52 havens, is plotted in Figure 4, is in DataFig5, and is counted among the 'BIS-reporting havens' in the Figure 6 data: the recount matches only when Bermuda is included. A minor inconsistency therefore exists (F16, item 5).

Audit notes

What reproduced. Reproduced: (1) Treaty dataset. Auditor A's port of the do-file matches the documented step counts and the published totals (1,025 events = 861 new + 164 domestic-law; 586/112/163 compliant/non-compliant/unreviewed; 1,090 overall). Auditor B's 7-event shortfall comes from B's port. (2) Figure 5 (-0.0384, SE 0.0139) and Figure 6 (-0.0062, SE 0.0189) slopes reproduce exactly from the workbook, as do the Figure 2-4 text numbers (2.7tn; 15.5% with a 2008 base; Jersey -4.2%; Hong Kong +2.4%; Cyprus +60% with 2 treaties; Guernsey -15% with 19) and footnotes 12, 13 and 15. Published Tables 1-5 equal the workbook. (3) Headline C1 on public BIS bilateral data for 8 of 13 havens, with three independent implementations: adjudicator -0.1118 (SE 0.057, p=0.050; 22,967 obs, 1,034 pairs), Auditor A -0.1121, Auditor B -0.1100 (published -0.1156, p=0.035). Also reproduced: new-treaty vs domestic-law split (-0.153/+0.113 vs -0.135/+0.016), growing post-signature dynamics, entry-into-force coding (-0.161), STD effect (-0.218 vs -0.216). Menkhoff & Miethe (2019) independently obtain -13% with JZ's design. (4) Swiss ESTV inputs to Figure 7 and Table A4 match the primary source (B checked 291 cells; adjudicator spot-checked France 2011); B closely reproduces the Table A4 null (-0.0255 vs -0.0253). Not reproduced: the Table 2 shifting coefficients (public data: coverage x (1-Signed) -0.0009 vs +0.0120; count coverage -0.0023 vs +0.0059); the Table 5 count coefficient (sign flips) and the significance of the share coefficient under haven-level inference; and the Figure 3 divergence (only ~2% in public data vs ~13% in the authors' 13-haven series). Exact replication of Tables 1-5 is impossible because the bilateral BIS data for Cayman, Cyprus, Macao, Malaysia and Panama (and the currency breakdowns) are confidential, as the README discloses.

Limits of this audit. The confidential BIS bilateral data (for Cayman, Cyprus, Macao, Malaysia and Panama, plus currency breakdowns) are unavailable. The public-data tests cover 8 of 13 havens (62.6% of 2007 deposits, ~74% of regression observations), are current vintage (2007q4 totals match the authors' to within 0.3%), and miss most non-Directive havens, so they cannot prove where the published shifting estimate comes from. Treaty events come from Auditor A's port of the authors' do-file, validated against its documented step counts and the published totals but not rebuilt from scratch by the adjudicator. The cohort and weighting analyses (F3) are subsample results with limited power. Footnote 9 (fiduciary share of Swiss BIS deposits), the Bank of England household share, BIS coverage rates and the Bettencourt details were not verified. The GAO report was read through a web-fetch summary, the 2012 working paper through a third-party mirror (HAL access failed), and the 2017 reprint was not checked. No formal published reply by the authors to these points was located. The compliance analysis cannot observe home-country tax returns, and the magnitude of the withholding-plus-declaration channel is unknown.