tte-surgery/docs/decisions.md

# Decisions log

Each decision is listed with the date and the reason for it. Settled decisions are changed only by adding a new entry below them, never by editing them.

## Round 1: framing (2026-10-27)

| # | Decision | Why |
|---|---|---|
| 1 | **Paper type: a "teaching" paper combined with a "threats" paper.** The backbone is that landmark, CCW and sequential trials answer three different clinical questions. The experimental factors are the threats specific to surgery. We do not claim that "TTE works". | Earlier simulations already show that the methods work. The new contribution is the surgical setting, plus showing what each estimand means in it. |
| 2 | **Primary outcome: MALE composite. Secondary outcomes: each component on its own.** | Matches the VESS 2026 abstract and practice in vascular surgery. The components show how outcomes that physicians generate (reintervention) behave differently from outcomes that come from disease (CLTI, amputation). The exact definition of MALE is still open (round 2). |
| 3 | **Include a simulated RCT arm in the same DGP** (ITT and naive per-protocol, with crossover, surgeon selection and small n). | Turns "why RCTs are hard in surgery" into numbers. |
| 4 | **Experimental factors:** (1) surgeon skill, (2) surgeon preference or style, (3) style acting on the outcome directly, (4) deferral and crossover driven by symptom progression, (5) unmeasured symptom severity. **Built in through the RCT arm:** (6) trial selection of high-skill surgeons. **Fixed feature of the DGP:** (7) treatment effect that changes over time. **Supplement / sensitivity analysis:** (8) unmeasured co-intervention (uptake of supervised exercise). | Keeps the main grid readable. |
| 5 | **Audience: surgeons. Target journal: JAMA Surgery.** Each method in the main text has a one-sentence clinical definition. The mathematics goes in the supplement. | The user is a surgical health-services researcher. Readers are clinicians. |
| 6 | **The RCT limitations are covered as narrative in the introduction plus one evidence table** (claudication RCTs: n, endpoint, crossover, recruitment shortfall). There is no separate systematic review. | Scope. |

## Round 2: the target trial (2026-10-27)

| # | Decision | Why |
|---|---|---|
| 7 | **MALE = CLTI, major amputation, open bypass, or any revascularization after the first.** The index PVI does not count as an event in either arm, whenever it happens. Total procedures per patient is a secondary outcome. **Supplement:** the Sorber "late PVI" definition (any PVI after month 6), to show how the outcome definition alone changes the answer. | Avoids an outcome that is asymmetric between arms by construction. Not counting the index procedure is defined the same way in both arms. |
| 8 | **Death is a competing event.** The primary measure is the 5-year cumulative incidence of MALE, reported as a risk difference. HR, and MALE-free survival (MALE or death), go in the supplement. | Medicare population with a mean age of about 76. Censoring at death would answer an unrealistic hypothetical question. |
| 9 | **Time zero is the new claudication diagnosis. Strategy A is PVI within 6 months.** A 3-month window is a supplement-only sensitivity analysis. | Matches Sorber 2023, VESS 2026, Dun 2025, and the guidelines. |
| 10 | **Strategy B, primary: no PVI in the first 6 months, then usual care.** **Strategy B, second (main text): no elective PVI unless a MALE-qualifying event occurs.** | Deferral vs abstention: "conservative" is not a single strategy (the ADAM/CAESAR lesson). |
| 11 | **Each method gets two errors: estimation bias (estimate vs its own estimand) and question gap (its own estimand vs the clinically relevant estimand). Total error is the sum of the two.** The clinically relevant reference is A vs B(primary) in all new claudicants under full adherence, which is the CCW estimand. | The teaching paper needs to separate "the method fails" from "the method answers another question". |
| 12 | **R, with a new self-contained package in this repo.** It follows the `tte-papers` conventions (YAML parameters, truths from simulating the interventions in large samples, Quarto, renv) but does not reuse its code. | The DGP differs (months, not hours; physicians as clusters). |

## Round 3: comparators and the simulated world (2026-10-27)

| # | Decision | Why |
|---|---|---|
| 13 | **Landmark:** month 6, among patients alive and MALE-free. Exposure is PVI in months 0–6. Outcome is MALE by month 60 *since diagnosis*. Adjust for baseline covariates plus measured covariates at month 6. | Same horizon as CCW. Mirrors the Sorber 2023 design, so the paper can say what that design estimated. |
| 14 | **Sequential trials:** monthly trials in months 0–5 among patients still eligible. **ITT is primary**; per-protocol (weighted) goes in the supplement. | Sequential ITT estimates "now vs maybe later" (deferral), in contrast to CCW's "within 6 months vs not". |
| 15 | **Physician-level variants, crossed with every method:** (1) ignore, (2) cluster-robust SE, (3) physician random intercept, (4) within-physician (FE or Mundlak). **Supplement:** (5) physician-preference IV. | Clustering fixes SEs, not bias. A random intercept does not remove confounding. FE can, but it risks Z-bias. The IV analysis is dubious when the surgeon also decides on reintervention. |
| 16 | **Monthly discrete time over 60 months. Pooled logistic hazard models.** | Standard for CCW and sequential trials, and simple. |
| 17 | **Measured:** age, sex, race, diabetes, CKD/dialysis, CHF, COPD, prior CVD, coded smoking, physician ID, practice setting (OBL/ASC vs hospital), calendar year, and a time-varying **proxy for symptom progression** (repeat vascular visits / noninvasive arterial testing). **Latent:** true symptom severity, ABI, true smoking, patient preference, surgeon skill, surgeon style. **Proxy strength is the dial for factor 5.** | What is "measured" means what is in Medicare claims. |
| 18 | **Two simulated RCTs:** a *realistic* one (n ≈ 240, hospital-based, top-third skill, low-preference surgeons) and an *ideal pragmatic* one (all physicians, large n). Crossover is **generated by the DGP** in both. Each is analysed by ITT and naive per-protocol. | Separates bias from surgeon selection and small n from bias due to crossover. |
| 19 | **Calibrate usual care to Medicare figures:** early PVI ≈ 3.2%; the physician distribution from Hicks 2020 (median 0%, range 0–58%, about 5.6% outliers); Sorber's crude late PVI and CLTI rates; age-appropriate mortality. **Truth scenarios:** null; harm (e.g. RD +5 points at 5 years); early benefit then late harm. | The null scenario shows pure bias. Calibration keeps the simulated world credible to surgeons. |

## Round 4: DGP core (2026-10-27)

| # | Decision | Why |
|---|---|---|
| 20 | **Mechanistic latent-severity engine.** Each patient has a latent severity that drifts with a patient-specific rate. Severity drives the CLTI hazard and the PVI decision. PVI lowers severity right away but adds a restenosis/progression hazard that depends on skill. | Benefit followed by harm, crossover in the conservative arm, and confounding by indication all come out of one mechanism. Truths are computed by simulating the interventions, so the extra complexity costs nothing there. |
| 21 | **Estimand: PVI done by the patient's own physician, averaged over the real physician mix.** The gap between the realistic RCT (top-third skill) and the population effect is reported as a finding (external validity), not as bias. | Skill modifies the effect of PVI. |
| 22 | **Skill and style are separate latent traits. Base correlation −0.3; comparison level 0.** | Sorber's outliers had more late PVI and CLTI. |
| 23 | **Factor 3 = high users have a lower threshold for repeat revascularization AND code CLTI more intensively.** The two are switched on together. | These are paths outside the treatment decision. Style acting through deferred PVI after month 6 is part of strategy B, not bias. |
| 24 | **Main analysis: one factor at a time from a clean base world, plus one "realistic, all threats on" world, in each of the 3 truth scenarios. Supplement: full factorial**, only if the one-at-a-time results suggest interactions. | Readability: one panel per threat. |
| 25 | **Each replicate has 50,000 patients and about 1,500 physicians, with lognormal panel sizes (median about 20). Supplement: 200,000 patients.** Number of replicates is set later, with ADEMP. | About 1,600 early-PVI patients per replicate, enough for CCW and FE positivity. |

## Round 5: analysis structure (2026-10-27)

| # | Decision | Why |
|---|---|---|
| 26 | **All observational methods use the same adjustment.** IP weights come from pooled logistic models on the measured covariates. Weighted cause-specific hazards for MALE and death are then combined into a standardized 5-year cumulative incidence. Landmark uses IPW at month 6. Sequential trials use IPW at each trial baseline. CCW uses artificial-censoring weights. The RCTs are analysed unadjusted (ITT) and as-treated (naive per-protocol). No matching and no doubly robust variants. | Differences between methods then reflect design and estimand only. Paper 1 already covered the choice of adjustment method. |
| 27 | **Performance measures:** estimation bias (against the method's own estimand), question gap, total error, empirical SE, model SE, coverage of the method's own estimand, RMSE, and MCSE for each. **The headline measure for clinicians is the false-alarm rate under the null** (CI excludes 0). | Separates two problems: whether the method fails, and whether it answers a different question. |
| 28 | **Variance:** cluster-robust sandwich SE plus the delta method in every replicate. A physician-cluster bootstrap (200 resamples) is run on about 100 replicates in the base and realistic worlds. | A full bootstrap at scale costs too much. Same approach as Paper 1. |
| 29 | **Replicates:** 100 at draft, 500 at final. MCSE is reported. | MCSE for coverage is about 1% at 500 replicates. |
| 30 | **Figure 1, the "estimand ladder":** one row per method. Each row shows a hollow dot for the method's own truth, a filled dot with interval for its estimate, and a line for the clinically relevant truth. Panels are worlds; colour is the physician variant. Shown for the null and harm scenarios. **Figure 2:** false-alarm rate by threat and physician variant. | One picture that shows both the question gap and the estimation bias. |
| 31 | **Next:** the analysis plan (md plus docx), then the table of claudication RCTs, then the build. The docs are committed on the branch `analysis-plan`. | |

## Round 6: negative controls (2026-10-27)

| # | Decision | Why |
|---|---|---|
| 32 | **Three negative controls.** **NCO-1:** hip fracture / pneumonia admission. Shares the latent patient frailty, severity and smoking, so it targets factor 5. **NCO-2:** varicose vein ablation by the same physician. Shares physician procedural style, so it targets factors 2–3. **NCE:** early low-value procedure (e.g. vein ablation) within 6 months, which shares both. All three are generated only from latent traits and never depend on PVI, so their true effect is exactly 0. They go through the same method × physician-variant pipeline as MALE. | Negative controls are the practical bias check available in real claims. CCW and landmark condition on survival, so the controls also need testing under each design. |
| 33 | **Main text (Figure 3 / Table 3):** how accurately the negative controls detect bias. A control is "flagged" if its CI excludes 0. Bias in the MALE risk difference counts if |bias| > 1 percentage point. Report sensitivity and specificity, plus how much bias can sit behind a clean negative control. **Supplement:** each control's own false-positive rate. | Answers a new, practical question: what does a clean negative control actually tell a surgeon? |

## Round 7: issues raised by drafting the plan (2026-10-27)

| # | Decision | Why |
|---|---|---|
| 34 | **Null scenario = sharp disease null** (Δ = κ = 0). The headline false-alarm rate is reported for the disease composite (CLTI or major amputation), whose true effect is exactly 0. MALE in the null world is reported as "harm" that comes from the outcome definition alone. | Repeat revascularization can only happen after an index PVI, so MALE cannot be exactly null. Showing that is a teaching point. |
| 35 | **Strength of each threat:** use an empirical anchor where one exists (style spread from Hicks 2020; deferred PVI from Sorber late-PVI rates; the early-PVI rate). Where none exists (skill, coding), set the threat so that on its own it moves the naive or landmark estimate by about 2–4 pp in the harm scenario. | Credible to surgeons while keeping each threat's effect visible. |
| 36 | **The NCE analysis is adjusted for early-PVI status.** The unadjusted version goes in the supplement. | Otherwise NCE ← style → PVI → MALE flags a real effect of PVI. |
| 37 | **Patient–physician sorting is not in the main grid.** It becomes one sensitivity world if time allows. | Scope. |
| — | **Fact (not a decision):** interim calibration target for all-cause mortality is about 35% at 5 years (Sigvant 2018 *EJVES*, PMID 29478910: 10-year mortality in IC 63%). To be replaced by the 5-year mortality in the authors' own Medicare cohort. | No published 5-year all-cause figure for Medicare claudicants was found. |