Files
tte-surgery/docs/analysis-plan-v1.0.md
T

30 KiB
Raw Blame History

Why surgical trials fail, and what each target trial emulation answers instead: a known-truth simulation of early peripheral vascular intervention for claudication

Use case: PVI within 6 months of a new claudication diagnosis, compared with initial conservative management; Medicare-like claims; 5-year major adverse limb events (MALE)

Analysis plan, draft v1.0, 2026-10-27. Every decision cited as "D#" is recorded, with its reason, in docs/decisions.md. The readings are in docs/literature/.

1. The paper in one paragraph

Whether to perform peripheral vascular intervention (PVI) early for claudication is one of the most common decisions in vascular surgery. It has never been tested in a randomized trial on outcomes that matter to the limb. The trials that do exist measured walking distance and quality of life in 56–240 patients, many recruited poorly, and in their conservative arms patients crossed over to intervention. Observational studies in Medicare suggest early PVI leads to more interventions and more chronic limb-threatening ischemia (CLTI). They use different designs, though (landmark, clone–censor–weight), and those designs answer different questions. We build one simulated Medicare-like population of claudicants and their physicians in which the true effect of early PVI is known by construction. In it we switch on, one at a time, the features that make surgical trials hard:

  • variation in surgeon skill;
  • surgeon practice style;
  • surgeons who also decide the outcome (reintervention, CLTI coding);
  • symptom-driven deferral and crossover;
  • unmeasured symptom severity.

We then run naive, landmark, sequential-trial and clone–censor–weight (CCW) analyses, with four ways of handling the physician, alongside the RCT that could realistically be run. Each method is scored against the question it actually answers and against the question a surgeon is asking. We also test whether negative controls can tell when the answer is wrong.

2. The gap

2.1 What is already known

  • Trials of claudication are small and short, and none measures hard outcomes. CLEVER (n = 111; planned 252), ERASE (212), IRONIC (158), MIMIC (127), SUPER (240, stopped early, 29% crossover in year 1) and OBACT (56 randomized of 434 screened) all used walking or QoL endpoints. Where follow-up was long, the benefit faded by 2–5 years (ERASE, IRONIC). The Cochrane review pooled no hard limb outcomes (Fakhry 2018).

  • Surgical RCTs fail for structural reasons. 21% of surgical RCTs stop early, mostly for poor recruitment (Chapman 2014). Surgical trials are discontinued more often than medical trials (43% vs 27%; Rosenthal 2015). Comparisons of surgery with non-surgery recruit about twice as slowly (Cook 2008). Crossover can make ITT uninterpretable (SPORT; Weinstein 2006). The intervention's effect depends on the operator (Birkmeyer 2013), and trial surgeons are selected (Rothwell 2005; Cutteridge 2025).

  • In timing trials, the "conservative" arm becomes delayed surgery. Examples: hernia watchful waiting reached 68% crossover in the long term (Fitzgibbons 2013); 62% of the ADAM surveillance arm were repaired (Lederle 2002); 85% of CAESAR by 54 months (Cao 2011).

  • Medicare evidence on early PVI. 3.2% of new claudicants have early PVI. The median physician rate is 0%, and 5.6% of physicians are outliers (Hicks 2020). Early PVI is associated with late PVI (22.5% vs 3.6%; aHR 6.89) and with CLTI (16.4% vs 7.8%) (Sorber 2023). A CCW emulation found more CLTI and MALE (Dun, VESS 2026). The Sorber design is in effect a 6-month landmark analysis.

  • Methods. Immortal time (Suissa 2008), landmark (Anderson 1983; Dafni 2011), CCW (Hernán 2018; Maringe 2020; grace periods: Wanis 2024) and sequential trials (Hernán 2008; Keogh 2023) are each well described. Duchesneau 2022 compared landmark and CCW against a plasmode truth in cancer surgery. Unmeasured confounding at the cluster level has been studied outside TTE (Li 2020; Cafri 2020). So has bias amplification from conditioning on an instrument (Myers 2011).

2.2 The gap this paper fills

No study has put landmark, sequential-trial and CCW analyses of one surgical timing question side by side, each scored against its own true estimand, in a world where the surgeon drives treatment, modifies its effect, and partly decides the outcome. The study should also run alongside the RCT that could realistically be done. So surgeons reading a Medicare TTE cannot tell:

  1. which question the analysis answered;
  2. how far that is from the question they care about;
  3. which surgery-specific threats could still bias it;
  4. whether a clean negative control means anything.

2.3 Why it matters

  • Claudication PVI is high-volume, increasingly done in office-based labs, and varies widely between physicians (Hicks 2020; Dun 2024). The question will be answered by claims analyses, not trials.

  • Journals now expect TTE (TARGET 2025). Readers in surgery need to know what each design estimates and where it breaks, in clinical language.

3. Aims and hypotheses

Aim Question Hypotheses Primary output
Aim 1. What each method answers In a clean world (all confounders measured, one physician type), how far is each method's own estimand from the clinically relevant one (question gap), and is each method unbiased for its own estimand? H1a: Each TTE method is unbiased for its own estimand. H1b: Landmark and sequential-trial ITT have large question gaps, while CCW-B1 has none by definition. H1c: The ITT of the realistic RCT is diluted towards the null by crossover, and is shifted by surgeon selection. Fig 1 (base panel), Table 2
Aim 2. Surgery-specific threats How does each threat change the estimation bias of each method × physician variant? H2a: Cluster-robust SEs fix coverage, not bias. H2b: A physician random intercept in the weight models removes only part of the confounding by physician; within-physician analysis (FE) removes it when style affects the outcome directly, but amplifies residual patient confounding when style acts only on treatment (Z-bias). H2c: Unmeasured severity biases every observational method; no physician variant fixes it. H2d: Crossover driven by deferral hurts landmark and RCT-ITT most. Fig 1 (threat panels), Fig 2, Table 3
Aim 3. Do negative controls detect the problem? When a method's MALE estimate is biased by more than 1 percentage point, how often does each negative control flag it? H3a: NCO-1 (patient-level) is sensitive to unmeasured severity or frailty. H3b: NCO-2 (physician-level) flags style variation even when style is not confounding, so its specificity is low. H3c: Substantial bias can sit behind a clean negative control. Fig 3, Table 4

Secondary (supplement): each MALE component separately; total procedures per patient; Sorber "late PVI" definition; HR scale; MALE-free survival; 3-month window; sequential trials per-protocol; preference IV; unmeasured supervised exercise; n = 200,000; full factorial if the one-at-a-time results suggest interactions.

4. Target trial specification

Component Target trial Emulation in the simulated claims
Eligibility Medicare beneficiaries ≥66 y with a new claudication diagnosis, no PVI, bypass, CLTI or amputation in the prior 12 months Same, at the diagnosis month (month 0)
Strategies A: PVI within 6 months of diagnosis. B1 (primary comparator): no PVI in months 0–5, then usual care. B2: no elective PVI at any time (PVI only after a MALE event) Observed PVI timing compared with each strategy
Assignment Randomized at diagnosis Exchangeable conditional on measured baseline and time-varying covariates (and, by variant, on the physician)
Time zero Diagnosis = eligibility = assignment Same
Follow-up Months 0–59. Ends at MALE, death or month 60 Same; no loss to follow-up (as in Medicare FFS without disenrollment)
Outcome MALE: first of CLTI, major amputation, open bypass, or any revascularization after the first (index) one (D7). Death is a competing event (D8) Same, from simulated claims
Causal contrast Per-protocol (sustained strategies) CCW with artificial-censoring weights
Measure 5-year cumulative incidence of MALE; risk difference A − B1 (and A − B2) Weighted cause-specific hazards → cumulative incidence

Clinically relevant estimand (the reference, D11): RD(A − B1) at month 60, among all new claudicants, with PVI done by each patient's own physician (D21), under full adherence.

5. Data-generating mechanism

5.1 Structure

Time is in months 0–59 (D16). Each replicate draws J ≈ 1,500 physicians and N = 50,000 patients (D25). Patients are assigned to physicians with lognormal panel sizes (median ≈ 20, long right tail).

Physicians (j).

  • Latent skill K_j ~ N(0, σ_K²) and latent style S_j ~ N(0, σ_S²), with corr(K, S) = ρ.
  • Practice setting OBL_j ∈ {0,1} is observed, with logit P(OBL_j = 1) = o₀ + o_S·S_j (high-style physicians are more often in office-based labs or ASCs).

Patients (i), at diagnosis.

  • Measured X_i: age, female, Black race, diabetes, CKD/dialysis, CHF, COPD, prior CVD, coded smoking, calendar year.
  • Latent frailty F_i, correlated with the measured comorbidities.
  • Latent true smoking (coded smoking is a misclassified version of it).
  • Latent initial severity Z_i0 = γ₀ + γ_X·X_i + e_i.
  • Latent progression rate r_i = r₀ + r_X·(diabetes, smoking) + u_i.

Latent severity process. Z_it = Z_i,t−1 + r_i + ε_it, so severity drifts upward at a patient-specific rate.

What a PVI does (the treatment mechanism, D20). At the month of PVI:

  • Relief. Severity drops by Δ (symptom relief, which lowers the near-term CLTI hazard).
  • Restenosis and progression. After PVI, the progression rate increases by κ·g(K_j), where g decreases with skill. This adds a skill-dependent hazard of repeat revascularization and of later CLTI.
  • Relief early and faster progression later produce the benefit-then-harm pattern (D4, factor 7). Δ and κ set the truth scenario (§5.3).

PVI decision, before the first PVI. The monthly hazard is

logit h_PVI(t) = α₀ + α_Z·Z_it + α_X·X_i + α_F·F_i + α_S·S_j + α_win·1[t ≥ 6]·(β_def + β_Z,def·Z_it)

  • Inside the window (months 0–5) the decision is driven by severity, which is confounding by indication.
  • After month 6, deferred PVI is governed by β_def and β_Z,def. This is factor 4.

Outcomes (monthly cause-specific hazards; pooled logistic form).

Process Depends on
True CLTI Z_it, X_i, F_i
Coded CLTI True CLTI + extra coding hazard ∝ S_j·1[factor 3] among patients with moderate Z_it
Major amputation Mainly after CLTI; small direct hazard from Z_it, diabetes, CKD
Open bypass (elective) Z_it; higher if a previous PVI failed
Repeat revascularization (only after the index PVI) Restenosis process (κ, K_j); lower threshold ∝ S_j·1[factor 3]
Death Age, X_i, F_i, Z_it (weak)
NCO-1: hip-fracture or pneumonia admission F_i, true smoking, age. Never PVI
NCO-2: varicose-vein ablation S_j, latent venous disease. Never PVI
NCE: low-value procedure (vein ablation) in months 0–5 S_j, F_i. Never affects any outcome

Measured vs latent (D17).

  • What the analyst sees: X_i, a claims frailty index, the physician ID, OBL_j, all coded events, and a time-varying severity proxy. The proxy is a monthly count of vascular visits and noninvasive arterial tests, P_it ~ Poisson(exp(p₀ + λ·Z_it)), plus its running sum.
  • What is latent: Z_it, r_i, F_i (the frailty index is its proxy), true smoking, K_j, S_j.

Table A. Variable roles

Block Causal role Clinical analogue Measured?
X Baseline confounders Age, sex, race, diabetes, CKD, CHF, COPD, prior CVD, coded smoking, year Yes
F Patient confounder (death, PVI candidacy, NCO-1) Frailty Through a frailty-index proxy
Z_t Time-varying confounder (PVI decision, CLTI) Walking limitation / ABI Through a visit/testing proxy, P_t (strength λ)
r Progression rate (prognostic) Diabetes/smoking-driven disease progression No
K Physician effect modifier (and confounder if ρ ≠ 0) Technical skill, durability of PVI No (physician ID only)
S Physician treatment driver; with factor 3 on, also an outcome driver Practice style / "high user" No (physician ID, OBL)
NCO-1, NCO-2, NCE Negative controls (true effect exactly 0) Hip fracture/pneumonia; vein ablation Yes
E (supplement only) Unmeasured co-intervention Supervised exercise uptake No

5.2 Worlds (one factor at a time, D24)

Every world is run under each of the three truth scenarios in §5.3.

World Skill (1) Style (2) Style → outcome (3) Deferral/crossover (4) Unmeasured severity (5)
W0 Base (clean) σ_K = 0 Modest σ_S Off Low, unrelated to severity Off: λ high, proxy ≈ Z
W1 Skill σ_K > 0, ρ = −0.3 (and ρ = 0) as W0 Off as W0 Off
W2 Style as W0 Strong σ_S, calibrated to the Hicks 2020 distribution Off as W0 Off
W3 Style → outcome as W0 Strong On as W0 Off
W4 Deferral as W0 as W0 Off High, severity-driven after month 6 Off
W5 Unmeasured severity as W0 as W0 Off as W0 On: λ low (proxy–Z correlation ≈ 0.4), frailty index noisy
WR Realistic On Strong On High On

W1 is run at both ρ = −0.3 and ρ = 0 (D22). WR is calibrated to the Medicare targets in §5.4.

5.3 Truth scenarios

Scenario Mechanism Target
Null (sharp disease null, D34) Δ = 0, κ = 0: PVI changes nothing about severity, CLTI, amputation, bypass or death RD(A − B1) = 0 exactly for the disease composite (CLTI or major amputation) and every disease component. MALE is still > 0, because only patients who have had a PVI can have a repeat revascularization: this is the mechanical effect of how the outcome is defined, and it is reported as a finding
Harm Δ small, κ moderate RD(A − B1) ≈ +5 percentage points
Benefit then harm Δ large, κ moderate Cumulative incidence curves cross: A lower before about month 18, higher at 60

5.4 Calibration targets for the usual-care world (WR)

Target Value Source
Early PVI (≤6 months) 3.2% Hicks 2020 (31208940); Sorber 2023 (37276171)
Physician early-PVI rate median 0%, range 0–58%, about 5.6% of physicians ≥2 SD outliers Hicks 2020
OBL/ASC share of PVI about 50% Dun 2024 (38958132)
Late PVI (>6 months, ~4.4 years) 22.5% after early PVI vs 3.6% without Sorber 2023
CLTI (~4.4 years) 16.4% vs 7.8% Sorber 2023
Mean age about 76 Dun 2025 (40782724)
All-cause mortality Interim target about 35% at 5 years (Swedish population cohort: 10-year mortality in IC 63% vs 27% reference). Replace with the 5-year mortality of the authors' own Medicare claudication cohort (Sorber 2023 / VESS 2026 data) Sigvant 2018 EJVES (29478910); cardiovascular-only 5-year mortality 14–16%: Baubeta Fridh 2018 BJS (29116654)
Amputation about 0.4%/year after revascularization for IC Baubeta Fridh 2017 (28797662)

Calibration solves for the intercepts and the key coefficients (α₀, α_S, σ_S, β_def, the outcome intercepts) by stochastic approximation on 10⁵ patients, as in tte-papers/simulations/01_calibrate.R.

5.5 Computing the truths

Truths come from simulating 10⁶ patients, reusing the physicians, under each intervention. The latent processes stay the same; only the PVI decision is overridden.

Estimand Intervention used for the truth
A Natural PVI hazard in months 0–4. Everyone still untreated, alive and MALE-free gets PVI at month 5 (the "natural grace" strategy that CCW with end-of-window censoring targets; Wanis 2024)
B1 PVI hazard = 0 in months 0–5, then natural
B2 PVI hazard = 0 at all times before MALE
Sequential-trial ITT For each m = 0–5, take the natural-course population eligible at m. Force PVI at m vs no PVI at m (natural course afterwards in both). Pool over m, weighted by the number eligible
Landmark No interventional definition, because the comparison is among survivors of the observed regime. Its own estimand is the oracle large-sample limit: the landmark IPW estimator on 10⁷ patients, with latent Z and F included in the weight model
Naive No own estimand. Only total error against the reference is reported
Realistic RCT ITT Simulate the trial population (hospital-based, top-third skill, low-style physicians; eligible patients), with assignment to A or B under natural adherence (refusal in A; severity-triggered crossover in B)
Ideal pragmatic RCT ITT Same, with all physicians

Truths are stored in simulations/truths/ as small committed CSVs, and each has its own Monte Carlo SE.

6. Estimands, in plain words (for the main text)

Method The question it answers
CCW, A vs B1 (the reference) "If every newly diagnosed claudicant had PVI within 6 months, compared with if every one waited 6 months and was then treated as usual, how different would 5-year MALE be?"
CCW, A vs B2 "…compared with if no one ever had an elective PVI?" (abstention, not deferral)
Sequential trials, ITT "For a patient who is still eligible this month, what does doing the PVI now change, compared with not now (perhaps later)?"
Landmark at 6 months "Among patients still alive and MALE-free at 6 months, how do those who happened to have a PVI differ from those who didn't?"
Naive "ever early PVI" No well-defined question (immortal time)
Realistic RCT, ITT "Among patients that selected, skilled, low-intervention surgeons are willing to randomize, what does assigning early PVI change, given that some patients refuse it and some controls cross over?"
Ideal pragmatic RCT, ITT Same, for all surgeons

7. Methods grid

Adjustment (D26). The same machinery is used everywhere. Inverse-probability weights come from pooled logistic models on the measured covariates (X, frailty index, P_t history, OBL, year). Weighted cause-specific pooled logistic hazards for MALE and death are fitted with flexible time terms and combined into standardized cumulative incidence at month 60. Weights are truncated at the 99.5th percentile in a supplementary analysis only.

Method Weights Notes
Naive Baseline PS for "PVI by month 6" Follow-up starts at month 0 for both groups, so immortal time is present
Landmark (D13) PS for "PVI in months 0–5" among patients at risk at month 6, using covariates up to month 6 Outcome counted through month 60
Sequential trials (D14) A PS for treatment at each trial baseline m = 0–5 Person-trials are stacked. ITT is primary; per-protocol (with IPCW for deviation) is in the supplement
CCW-B1, CCW-B2 Clones are artificially censored at deviation, with IPCW from the PVI-hazard model A clones untreated at month 5 are censored at the end of the window
Realistic and ideal RCT None ITT by assigned arm; naive per-protocol by treatment received

Physician variants (D15), crossed with every observational method. The physician enters through the weight models, because that is where confounding is controlled.

Variant Implementation
V1 Ignore No physician terms; model-based SE
V2 Cluster-robust As V1, with the sandwich SE clustered on the physician
V3 Random intercept Physician random intercept in the weight models (glmmTMB/lme4); cluster-robust SE
V4 Within-physician Physician fixed effects in the weight models. Physicians with no variation in early PVI are dropped, and the change to the estimand that this causes is reported. Mundlak (physician means) is the fallback if FE is too slow. Cluster-robust SE
V5 Preference IV (supplement) Instrument = the physician's early-PVI rate among the physician's other patients; two-stage residual inclusion

Negative controls (D32). NCO-1 and NCO-2 replace MALE as the outcome in every method × variant. The NCE replaces early PVI as the exposure in the CCW and landmark designs. The NCE models adjust for early-PVI status (D36). Without that adjustment, the path NCE ← style → PVI → MALE would make the NCE "detect" a real effect of PVI. The unadjusted NCE analysis is in the supplement.

8. Simulated RCTs (D18)

Realistic RCT Ideal pragmatic RCT
Physicians Hospital-based (OBL = 0), top third of skill, bottom half of style All physicians
Patients Eligible patients of those physicians; a constant 30% consent All eligible patients
n 240 (120 per arm) 50,000
Arm A PVI at the next scheduled month in 0–2. Refusal 10% Same
Arm B No PVI in months 0–5 unless severity crosses the "lifestyle-limiting" threshold (crossover generated by the DGP); usual care afterwards Same
Analyses ITT (unadjusted cumulative incidence) and naive as-treated Same

Running replicates of the 240-patient trial also gives its power and CI width, which is the "small trial" result.

9. Performance measures and simulation size (D27–D29)

Notation. For method m with estimate θ̂_m, its own estimand θ_m, and the clinically relevant estimand θ*:

Measure Definition
Estimation bias E[θ̂_m] − θ_m
Question gap θ_m − θ*
Total error E[θ̂_m] − θ* (the sum of the two above)
Empirical SE SD of θ̂_m across replicates
Model SE Mean of the estimated SEs
Coverage Coverage of θ_m by the 95% CI
RMSE Against θ_m and against θ*
False-alarm rate (headline) In the null scenario, the proportion of replicates whose 95% CI excludes 0 for the disease composite (CLTI or major amputation), whose true effect is exactly 0 (D34)

MCSE is reported for every measure (Morris 2019).

  • Variance (D28). A sandwich SE clustered on the physician, with the delta method for the cumulative-incidence RD. Validated by a physician-cluster bootstrap (200 resamples) on 100 replicates in W0 and WR.
  • Replicates (D29). 100 at draft scale, 500 at final. With 500 replicates, the MCSE of coverage is about 1% and the MCSE of bias is about 0.045 × empirical SE.

10. Negative-control diagnostic analysis (Aim 3, D33)

For each world × scenario × method × variant × replicate, record:

  • Flag: the negative control's 95% CI excludes 0.
  • Truly biased: |θ̂_m − θ_m| > 1 percentage point. Also |θ̄_m − θ_m| > 1 pp at the cell level.

Reported:

  • the sensitivity and specificity of each negative control, and of "any negative control", for detecting bias;
  • the distribution of |bias| among replicates with a clean negative control ("how much can hide");
  • in the supplement, each negative control's false-positive rate.

11. Mock tables

  • Table 1 (introduction). Randomized trials of revascularization for claudication. Trial, year, n (planned vs actual), comparison, primary endpoint, follow-up, crossover, whether the trial stopped early, whether there was a hard limb outcome. See Appendix A.

  • Table 2. What each method answers (W0, all three scenarios). Rows: methods. Columns: plain-language estimand, own truth θ_m, question gap, estimation bias, coverage.

  • Table 3. Bias by threat. Rows: method × physician variant. Columns: W0–W5, WR. Cells: estimation bias (MCSE), shaded by |bias| > 1 pp.

  • Table 4. Negative controls as bias detectors. Rows: NCO-1, NCO-2, NCE, "any". Columns: sensitivity, specificity, and the median |bias| when the negative control is clean, by world.

12. Mock figures

  • Fig 1. Estimand ladder (D30). One row per method. Each row shows the method's own truth (hollow dot), its mean estimate and 95% interval across replicates (filled dot, coloured by physician variant), and the clinically relevant truth (vertical line). Panels: W0, W1–W5, WR. Two versions: null and harm.

  • Fig 2. False-alarm rate in the null scenario (disease composite). Heat map of method × variant (rows) against world (columns). The 5% nominal level is marked. An inset shows the MALE estimate in the same null world: "harm" produced purely by how the outcome is defined.

  • Fig 3. What a clean negative control is worth. For each method, |bias| plotted for flagged and clean replicates, by world.

  • Fig S. The DGP as a DAG, and cumulative-incidence curves under A, B1 and B2 for each scenario.

13. Position relative to the literature

Prior work What it showed What this paper adds
CLEVER, ERASE, IRONIC, SUPER; Fakhry 2018 Small trials of walking and QoL, with crossover The trial that could realistically be run, simulated and compared with the question surgeons ask
Sorber 2023; Hicks 2020; Dun 2025/2026 Medicare associations; physician outliers; CCW emulation What those designs actually estimated, and which threats remain
Duchesneau 2022 CEBP Landmark vs CCW against a plasmode truth, in cancer surgery Adds sequential trials, RCTs and the physician level; estimand-specific truths
Keogh 2023; Hernán 2008/2018; Wanis 2024 Sequential trials, CCW, grace periods in theory and simulation The difference between deferral and abstention shown in a surgical question
Li 2020 SMMR; Cafri 2020 Biom J Unmeasured cluster confounding; within-surgeon matching Inside TTE designs, with the surgeon acting on treatment, effect and outcome
Myers 2011; Brookhart 2006 Z-bias; preference IV Shown for physician style in claims
Paper 1 (tte-papers) Design matters more than the adjustment method (ICU) A surgical setting; estimand gaps; negative controls as diagnostics

14. Out of scope

  • PS matching and doubly robust estimators (Paper 1 territory).
  • Data-quality problems (Paper 2 territory): coding error beyond factor 3, missing data, disenrollment.
  • Patient–physician sorting, where sicker patients see particular physicians. Patients are assigned to physicians at random (D37); sorting is a possible single sensitivity world.
  • Cost-effectiveness.

15. Practicalities

Item Plan
Software R (4.6.1). A new package pvisim/ in this repo with parameters in YAML, a simulator, truths, designs, estimators and metrics (D12). Uses fastglm/speedglm for pooled logistic, sandwich, glmmTMB, data.table, arrow, furrr. Quarto reports; renv
Validation Before any runs: (a) every method is unbiased for its own estimand in W0 at n = 10⁶ (within MCSE); (b) a closed-form check of the CCW truth in a special case with no time-varying confounding; (c) negative-control truths are exactly 0 by construction (tests)
Compute 7 worlds (+ W1 at ρ = 0) × 3 scenarios × 500 replicates. Each replicate fits about 7 methods × 4 variants × (MALE + 3 negative controls). V3/V4 on person-month data are the bottleneck. Profile at the draft stage and switch to Mundlak if needed
Sequence 1) parameters YAML → 2) simulator + tests → 3) calibration → 4) truths → 5) estimators, validated at large n → 6) draft runs (100 replicates) → 7) report → 8) final runs → 9) manuscript
Reporting ADEMP (Morris 2019); TARGET 2025 items for the emulated trial; code and seeds public
Target journal JAMA Surgery (Original Investigation). The mathematics and the full grid go in the supplement

16. Decisions resolved after the first draft (D34–D37)

  1. Null (D34). Sharp disease null. The headline false-alarm rate is reported for the disease composite (CLTI or major amputation). MALE in the null world shows "harm" that comes purely from the outcome definition.
  2. Strength of each threat when switched on (D35). Use the empirical quantity where one exists (physician-style spread from the Hicks 2020 outlier distribution; deferred PVI from Sorber's late-PVI rates; early-PVI rate). Where none exists (skill, coding intensity), set the threat so that on its own it moves the naive or landmark estimate by about 2–4 percentage points in the harm scenario.
  3. NCE (D36). The NCE analysis is adjusted for early-PVI status. The unadjusted version goes in the supplement as a warning.
  4. Patient–physician sorting (D37). Not in the main grid. One sensitivity world, if time allows.

Still to do: replace the interim mortality target (§5.4) with the 5-year mortality of the authors' own Medicare cohort.


Appendix A. Draft Table 1: randomized trials of revascularization for intermittent claudication

Trial n (planned → randomized) Comparison Primary endpoint Longest follow-up Crossover / adherence Stopped early / under-recruited Hard limb outcome?
CLEVER (Murphy 2012, 2015) 252 → 111 OMC vs OMC+SE vs OMC+stent (aortoiliac) Peak walking time, 6 months 18 months (79 completed) — Under-recruited (reduced to 3 arms) No
ERASE (Fakhry 2015; Klaphake 2022) 212 Endovascular + SET vs SET Maximum walking distance, 12 months ~5.4 years (post hoc) SET arm: later revascularization HR 2.5 No No (procedures reported)
IRONIC (Nordanstig 2014; Djerf 2020) 158 Revascularization + BMT + exercise vs BMT + exercise SF-36, 1 year 5 years (benefit lost) — Single centre No
MIMIC (Greenhalgh 2008) 93 + 34 PTA added to SE + BMT Walking distance, 24 months 24 months — Small No
SUPER (Koelemay 2022) 240 SET vs endovascular (iliac) MWD + VascuQol, 1 year ~5.5 years 29% crossover by year 1; 50% SET compliance Stopped early No
Mazari (2012, 2017) 178 PTA vs SEP vs both (fem-pop) Walking/QoL, 12 months 5.2 years — — No (reinterventions reported)
Spronk (2009) 151 Endovascular vs SET Rutherford, 6–12 months 12 months — — No
OBACT (Nylaende 2007) 434 screened → 56 OMT ± PTA QoL, ABI, walking, 2 years 2 years — 56 of 434 screened No

Missing cells are to be filled from the full texts (crossover is not reported in several abstracts).