tte-surgery/docs/decisions.md

Decisions log

Each decision is listed with the date and the reason for it. Settled decisions are changed only by adding a new entry below them, never by editing them.

Round 1: framing (2026-10-27)

#	Decision	Why
1	Paper type: a "teaching" paper combined with a "threats" paper. The backbone is that landmark, CCW and sequential trials answer three different clinical questions. The experimental factors are the threats specific to surgery. We do not claim that "TTE works".	Earlier simulations already show that the methods work. The new contribution is the surgical setting, plus showing what each estimand means in it.
2	Primary outcome: MALE composite. Secondary outcomes: each component on its own.	Matches the VESS 2026 abstract and practice in vascular surgery. The components show how outcomes that physicians generate (reintervention) behave differently from outcomes that come from disease (CLTI, amputation). The exact definition of MALE is still open (round 2).
3	Include a simulated RCT arm in the same DGP (ITT and naive per-protocol, with crossover, surgeon selection and small n).	Turns "why RCTs are hard in surgery" into numbers.
4	Experimental factors: (1) surgeon skill, (2) surgeon preference or style, (3) style acting on the outcome directly, (4) deferral and crossover driven by symptom progression, (5) unmeasured symptom severity. Built in through the RCT arm: (6) trial selection of high-skill surgeons. Fixed feature of the DGP: (7) treatment effect that changes over time. Supplement / sensitivity analysis: (8) unmeasured co-intervention (uptake of supervised exercise).	Keeps the main grid readable.
5	Audience: surgeons. Target journal: JAMA Surgery. Each method in the main text has a one-sentence clinical definition. The mathematics goes in the supplement.	The user is a surgical health-services researcher. Readers are clinicians.
6	The RCT limitations are covered as narrative in the introduction plus one evidence table (claudication RCTs: n, endpoint, crossover, recruitment shortfall). There is no separate systematic review.	Scope.

Round 2: the target trial (2026-10-27)

#	Decision	Why
7	MALE = CLTI, major amputation, open bypass, or any revascularization after the first. The index PVI does not count as an event in either arm, whenever it happens. Total procedures per patient is a secondary outcome. Supplement: the Sorber "late PVI" definition (any PVI after month 6), to show how the outcome definition alone changes the answer.	Avoids an outcome that is asymmetric between arms by construction. Not counting the index procedure is defined the same way in both arms.
8	Death is a competing event. The primary measure is the 5-year cumulative incidence of MALE, reported as a risk difference. HR, and MALE-free survival (MALE or death), go in the supplement.	Medicare population with a mean age of about 76. Censoring at death would answer an unrealistic hypothetical question.
9	Time zero is the new claudication diagnosis. Strategy A is PVI within 6 months. A 3-month window is a supplement-only sensitivity analysis.	Matches Sorber 2023, VESS 2026, Dun 2025, and the guidelines.
10	Strategy B, primary: no PVI in the first 6 months, then usual care. Strategy B, second (main text): no elective PVI unless a MALE-qualifying event occurs.	Deferral vs abstention: "conservative" is not a single strategy (the ADAM/CAESAR lesson).
11	Each method gets two errors: estimation bias (estimate vs its own estimand) and question gap (its own estimand vs the clinically relevant estimand). Total error is the sum of the two. The clinically relevant reference is A vs B(primary) in all new claudicants under full adherence, which is the CCW estimand.	The teaching paper needs to separate "the method fails" from "the method answers another question".
12	R, with a new self-contained package in this repo. It follows the tte-papers conventions (YAML parameters, truths from simulating the interventions in large samples, Quarto, renv) but does not reuse its code.	The DGP differs (months, not hours; physicians as clusters).

Round 3: comparators and the simulated world (2026-10-27)

#	Decision	Why
13	Landmark: month 6, among patients alive and MALE-free. Exposure is PVI in months 0–6. Outcome is MALE by month 60 since diagnosis. Adjust for baseline covariates plus measured covariates at month 6.	Same horizon as CCW. Mirrors the Sorber 2023 design, so the paper can say what that design estimated.
14	Sequential trials: monthly trials in months 0–5 among patients still eligible. ITT is primary; per-protocol (weighted) goes in the supplement.	Sequential ITT estimates "now vs maybe later" (deferral), in contrast to CCW's "within 6 months vs not".
15	Physician-level variants, crossed with every method: (1) ignore, (2) cluster-robust SE, (3) physician random intercept, (4) within-physician (FE or Mundlak). Supplement: (5) physician-preference IV.	Clustering fixes SEs, not bias. A random intercept does not remove confounding. FE can, but it risks Z-bias. The IV analysis is dubious when the surgeon also decides on reintervention.
16	Monthly discrete time over 60 months. Pooled logistic hazard models.	Standard for CCW and sequential trials, and simple.
17	Measured: age, sex, race, diabetes, CKD/dialysis, CHF, COPD, prior CVD, coded smoking, physician ID, practice setting (OBL/ASC vs hospital), calendar year, and a time-varying proxy for symptom progression (repeat vascular visits / noninvasive arterial testing). Latent: true symptom severity, ABI, true smoking, patient preference, surgeon skill, surgeon style. Proxy strength is the dial for factor 5.	What is "measured" means what is in Medicare claims.
18	Two simulated RCTs: a realistic one (n ≈ 240, hospital-based, top-third skill, low-preference surgeons) and an ideal pragmatic one (all physicians, large n). Crossover is generated by the DGP in both. Each is analysed by ITT and naive per-protocol.	Separates bias from surgeon selection and small n from bias due to crossover.
19	Calibrate usual care to Medicare figures: early PVI ≈ 3.2%; the physician distribution from Hicks 2020 (median 0%, range 0–58%, about 5.6% outliers); Sorber's crude late PVI and CLTI rates; age-appropriate mortality. Truth scenarios: null; harm (e.g. RD +5 points at 5 years); early benefit then late harm.	The null scenario shows pure bias. Calibration keeps the simulated world credible to surgeons.

Round 4: DGP core (2026-10-27)

#	Decision	Why
20	Mechanistic latent-severity engine. Each patient has a latent severity that drifts with a patient-specific rate. Severity drives the CLTI hazard and the PVI decision. PVI lowers severity right away but adds a restenosis/progression hazard that depends on skill.	Benefit followed by harm, crossover in the conservative arm, and confounding by indication all come out of one mechanism. Truths are computed by simulating the interventions, so the extra complexity costs nothing there.
21	Estimand: PVI done by the patient's own physician, averaged over the real physician mix. The gap between the realistic RCT (top-third skill) and the population effect is reported as a finding (external validity), not as bias.	Skill modifies the effect of PVI.
22	Skill and style are separate latent traits. Base correlation −0.3; comparison level 0.	Sorber's outliers had more late PVI and CLTI.
23	Factor 3 = high users have a lower threshold for repeat revascularization AND code CLTI more intensively. The two are switched on together.	These are paths outside the treatment decision. Style acting through deferred PVI after month 6 is part of strategy B, not bias.
24	Main analysis: one factor at a time from a clean base world, plus one "realistic, all threats on" world, in each of the 3 truth scenarios. Supplement: full factorial, only if the one-at-a-time results suggest interactions.	Readability: one panel per threat.
25	Each replicate has 50,000 patients and about 1,500 physicians, with lognormal panel sizes (median about 20). Supplement: 200,000 patients. Number of replicates is set later, with ADEMP.	About 1,600 early-PVI patients per replicate, enough for CCW and FE positivity.

Round 5: analysis structure (2026-10-27)

#	Decision	Why
26	All observational methods use the same adjustment. IP weights come from pooled logistic models on the measured covariates. Weighted cause-specific hazards for MALE and death are then combined into a standardized 5-year cumulative incidence. Landmark uses IPW at month 6. Sequential trials use IPW at each trial baseline. CCW uses artificial-censoring weights. The RCTs are analysed unadjusted (ITT) and as-treated (naive per-protocol). No matching and no doubly robust variants.	Differences between methods then reflect design and estimand only. Paper 1 already covered the choice of adjustment method.
27	Performance measures: estimation bias (against the method's own estimand), question gap, total error, empirical SE, model SE, coverage of the method's own estimand, RMSE, and MCSE for each. The headline measure for clinicians is the false-alarm rate under the null (CI excludes 0).	Separates two problems: whether the method fails, and whether it answers a different question.
28	Variance: cluster-robust sandwich SE plus the delta method in every replicate. A physician-cluster bootstrap (200 resamples) is run on about 100 replicates in the base and realistic worlds.	A full bootstrap at scale costs too much. Same approach as Paper 1.
29	Replicates: 100 at draft, 500 at final. MCSE is reported.	MCSE for coverage is about 1% at 500 replicates.
30	Figure 1, the "estimand ladder": one row per method. Each row shows a hollow dot for the method's own truth, a filled dot with interval for its estimate, and a line for the clinically relevant truth. Panels are worlds; colour is the physician variant. Shown for the null and harm scenarios. Figure 2: false-alarm rate by threat and physician variant.	One picture that shows both the question gap and the estimation bias.
31	Next: the analysis plan (md plus docx), then the table of claudication RCTs, then the build. The docs are committed on the branch analysis-plan.

Round 6: negative controls (2026-10-27)

#	Decision	Why
32	Three negative controls. NCO-1: hip fracture / pneumonia admission. Shares the latent patient frailty, severity and smoking, so it targets factor 5. NCO-2: varicose vein ablation by the same physician. Shares physician procedural style, so it targets factors 2–3. NCE: early low-value procedure (e.g. vein ablation) within 6 months, which shares both. All three are generated only from latent traits and never depend on PVI, so their true effect is exactly 0. They go through the same method × physician-variant pipeline as MALE.	Negative controls are the practical bias check available in real claims. CCW and landmark condition on survival, so the controls also need testing under each design.
33	Main text (Figure 3 / Table 3): how accurately the negative controls detect bias. A control is "flagged" if its CI excludes 0. Bias in the MALE risk difference counts if	bias

Round 7: issues raised by drafting the plan (2026-10-27)

#	Decision	Why
34	Null scenario = sharp disease null (Δ = κ = 0). The headline false-alarm rate is reported for the disease composite (CLTI or major amputation), whose true effect is exactly 0. MALE in the null world is reported as "harm" that comes from the outcome definition alone.	Repeat revascularization can only happen after an index PVI, so MALE cannot be exactly null. Showing that is a teaching point.
35	Strength of each threat: use an empirical anchor where one exists (style spread from Hicks 2020; deferred PVI from Sorber late-PVI rates; the early-PVI rate). Where none exists (skill, coding), set the threat so that on its own it moves the naive or landmark estimate by about 2–4 pp in the harm scenario.	Credible to surgeons while keeping each threat's effect visible.
36	The NCE analysis is adjusted for early-PVI status. The unadjusted version goes in the supplement.	Otherwise NCE ← style → PVI → MALE flags a real effect of PVI.
37	Patient–physician sorting is not in the main grid. It becomes one sensitivity world if time allows.	Scope.
—	Fact (not a decision): interim calibration target for all-cause mortality is about 35% at 5 years (Sigvant 2018 EJVES, PMID 29478910: 10-year mortality in IC 63%). To be replaced by the 5-year mortality in the authors' own Medicare cohort.	No published 5-year all-cause figure for Medicare claudicants was found.

Round 8: scope (2026-10-27)

#	Decision	Why
38	No data-quality aim. The paper keeps the three aims of the plan. Claims data problems such as time-zero misdating, physician attribution, laterality and Medicare Advantage switching are not modelled.	Focus. Papers 1 and 2 of tte-papers have stopped.
39	Paper 1's comparison of adjustment methods and covariate vibration stay out (see D26).	This paper is about which question each design answers.

Build notes: simulator and calibration (2026-10-27)

#	Decision	Why
40	Factor 5 (unmeasured confounding by indication): "off" means the analyst sees true severity Z_t, true frailty and true smoking. "On" means the analyst sees only claims proxies: a 3-month visit/testing count for severity, a frailty index with reliability 0.4, and coded smoking (sensitivity 0.6).	In the clean worlds every confounder must be measured, so each method can be checked as unbiased for its own estimand.
41	Each world's intercepts are re-solved by calibration (early PVI, deferred PVI, repeat revascularization, CLTI, death) so that every world matches the same Medicare targets. A world differs from W0 only in the threat it switches on.	A fair comparison between worlds.
42	Deferral: in W0–W3, W5 and WR, deferred PVI matches Sorber (3.6% late PVI among patients without early PVI). W4 is calibrated to 15%, which is trial-like crossover, and in W4 deferral is driven by severity. In WR, deferral is at the Sorber rate but driven by severity. PVI after CLTI was lowered to about 1% per month, so most late PVI is elective.	At the Medicare rate, the deferral threat would be almost invisible. Trials (SUPER 29% in year 1) show much more crossover.
43	Physician style coefficient 1.6 (W2, W3, WR). Among physicians with ≥11 patients this gives a median early-PVI rate of 0.8%, a maximum of 75%, and 4.7% of physicians ≥2 SD outliers who perform 38% of early PVIs. Hicks 2020 reported 0%, 58% and 5.6%.	Close enough to Hicks 2020. Not tuned further.
44	Within a month the order is: severity update → outcome events → death → elective PVI decision. A MALE event and death in the same month count the MALE event first.	Matches discrete-time pooled logistic analysis.

Build notes: truths, estimators, validation (2026-10-27)

#	Decision	Why
45	Scenario effects, tuned in W0 against the disease composite: harm (relief 0.3, kappa 0.039) gives RD(A−B1) +5.1 pp. Benefit-then-harm (relief 2.0, kappa 0.075) gives −1.2 pp at 24 months and +3.1 pp at 60 months. The same mechanism is used in every world. Where skill varies (W1, WR) the true effects are larger (+7 pp), because exp(−skill) is convex: on average a PVI is done worse than one at average skill.	D34 sets the headline on the disease composite. MALE carries an extra +7 to +15 pp that comes purely from the outcome definition.
46	Truths for the negative controls are taken from the truths table, not assumed to be 0. In the non-null scenarios PVI changes death, so a negative-control outcome's 5-year risk moves slightly through the competing risk (e.g. NCO-1 −0.08 pp). In the null scenario the effect is exactly 0.	Honest scoring.
47	The own estimand of landmark and naive is estimated within each replicate by an oracle run: the same estimator with exact severity, true frailty and smoking, and the latent skill and style. Estimation bias is the paired difference (estimate − oracle).	A large-n limit was too noisy: the landmark MCSE at n = 10⁵ was 2.8 pp.
48	The PVI model is fitted separately inside the window (month indicators) and in the deferred period (spline in month).	A pooled model misspecified the window probabilities that the CCW and sequential weights depend on.
49	V2 is the V1 fit with a physician-cluster-robust SE. Under V4, physicians without an estimable fixed effect in a phase get probability 0, and the V4 population is restricted to physicians with variation in early PVI.	Same point estimate by construction (D15).
50	Validation (W0, harm, 40 replicates of 50,000): CCW A−B1 is +5.12 vs a truth of +5.11 on the disease composite (z = 0.0); sequential ITT is +5.83 vs +5.71 (z = 0.6); MALE and NCO-1 have	z
51	Speed and scale. The PVI model is fitted once per replicate and variant, and shared by CCW and the sequential trials. CCW clones and weights are built once for all outcomes. The deferred-period PVI model uses every PVI plus a fixed 10% sample of the other person-months, with offset log(10). On W0 validation the estimates are identical to 4 decimals. The main designs take about 39 s per replicate instead of about 130 s. Scale: n = 50,000 stays. Draft is 100 replicates; final is 200 replicates (replacing 500 in D29), and the final run extends the draft's replicates 1–100. Total about 12 hours on 11 cores.	User asked for a faster, smaller run. The sample size is not reduced, because CCW's strategy A already depends on about 250 patients treated in month 5 at n = 50,000. With 200 replicates, MCSE of bias ≈ empirical SE / 14 (CCW about 0.2 pp) and MCSE of coverage ≈ 1.5%.

Draft run (2026-10-28)

#	Decision	Why
52	Draft run done: 8 worlds × 3 scenarios × 100 replicates of 50,000 patients; 9.4 h on 11 cores; 0 errors. 16 of 288,000 estimates are NA (sequential trials, V4, 4 replicates) and are dropped. Summary: simulations/04_summarise.R writes results/summary/. Report: reports/draft-report.Rmd. Two open questions: (Q1) the landmark propensity model adjusts for month-5 severity, a post-PVI mediator, which drives the landmark question gap (W1 own truth about 20 pp vs a reference of 7.2 pp); with baseline covariates only, the 20-replicate oracle mean is 8.1. (Q2) NCO-1 has no severity term in the DGP, contrary to D32, so it cannot detect W5/WR bias. The bootstrap check (D28) waits on Q1/Q2.	Surfaced by the draft results.
53	Answers to D52 (user, 2026-10-28). (Q1) The primary landmark propensity score uses baseline covariates only (diagnosis covariates and month-0 severity). The version that also adjusts for month-5 severity becomes landmark_m5 (supplement). It conditions on a mediator, because a PVI in months 0–5 has already changed that severity. Both versions get their own oracle. (Q2) NCO-1 now depends on severity (coefficient 0.3), using the patient's PVI-free severity path (the same baseline, rate and monthly noise, with no relief and no kappa). That way NCO-1 shares the confounder but still cannot be affected by PVI (only through death as a competing risk, D46). Intercept −6.75 keeps 5-year NCO-1 incidence at about 9.3%. This is the best case for a patient-level negative control, and the paper says so. Bootstrap (D28, reduced): W0 and WR × null and harm, replicates 1–20 of the main run, 50 physician-cluster resamples each, V1 point estimates, about 8 s per resample. Compare the bootstrap SD with the model SE (patient and cluster) and the empirical SE. The rerun goes to results/main/n50k_v2; the first draft stays in n50k. Truths are recomputed (NCO-1 changed).	Landmark analyses in practice adjust for baseline covariates. D32 intended NCO-1 to share severity.

Paper framing (2026-10-28)

#	Decision	Why
54	The paper's question: "Can target trial emulation overcome the problems that make randomized trials hard in surgery?" Early PVI for claudication is the worked example. Medicare only supplies realistic simulation inputs, and the paper makes no claims about Medicare findings or the authors' earlier papers. The structure is a scorecard: each surgical-RCT barrier (small/stopped early, short follow-up/soft endpoints, crossover, selected surgeons, practice style, unmeasured severity) against whether TTE solves it, checked against the known truth. Aims 1–3 stay. Aim 1 = "TTE is not one method", Aim 2 = the scorecard rows, Aim 3 = whether the one unfixable problem can be detected. The MALE/repeat-revascularization point is secondary, general outcome-definition guidance. The Medicare "dial" idea is dropped. Novelty (quick PubMed check, 2026-10-28): there are TTE guides for surgeons (e.g. JBJS Rev 2026) and immortal-time design comparisons (Duchesneau 2022 CEBP), but no search hit for a known-truth simulation comparing TTE with a feasible surgical RCT under surgeon-level threats.	User: the aim is to test whether TTE can solve RCT challenges in surgery, without arguing against the authors' earlier results.
55	Three additions from literature review 3 (user, 2026-10-28), for the overnight run. (1) New world WE, "EHR/registry-like": the same DGP and calibration as W0, but the analyst sees severity with measurement error (monthly Z_t + N(0, 0.75²), correlation with true severity about 0.8), a frailty index with reliability 0.7, and true smoking. It sits between W0 (exact) and W5 (claims proxy) and answers the claims-vs-EHR row of Table 1. (2) Non-random consent in the realistic trial: mean 30%, OR 0.7 per 10 years of age and per SD of frailty (Wasmann 2019, King 2005). The trial's ITT truth is weighted by consent probability. (3) Trial horizons 12, 24 and 60 months: ITT and as-treated at each. Truths rct_<design>_itt_h12/_h24. The reference at the short horizons is RD(A − B1) at that month, from the risk curves. Learning curve: not simulated; limitation (user).	Completes Table 1 rows 3, 5 and 17.
56	Primary outcome changed to limb MALE: the first of coded CLTI, major amputation or open bypass (user, 2026-10-28). D7's MALE (which also counts any revascularization after the first) moves to the supplement as "MALE including repeat revascularization". Repeat revascularization is reported on its own as a physician-driven outcome. Limb MALE also replaces the disease composite (CLTI or amputation) as the headline (D34); the disease composite stays as a secondary outcome. Negative-control bias (Aim 3) is now measured on limb MALE. Clean-world truths: 0 under the null (exactly); +6.2 under harm; −0.7 at 12 months, −1.7 at 24 and +3.6 at 60 under benefit-then-harm. Acute limb ischemia is not simulated (rare in claudication; limitation).	Standard MALE definitions (Conte 2009 JVS 19897335; Nordanstig 2024 EJVES 39121905) are built for follow-up after a revascularization: reintervention exists only for treated patients. D7 argued that excluding the index PVI in both arms made the outcome symmetric. It does not: under A everyone is at risk of a repeat for about 55 months, under B1 about 3.6% are. In the clean world under the null, repeat revascularization is 18.6% vs 0.7%, giving +14 pp of MALE from the definition alone.