tte-surgery/docs/literature/03-rct-challenges-and-tte.md

Literature review 3: challenges of surgical RCTs, and whether TTE solves them

Merged from three parallel searches (2026-10-28). The evidence, with verified PMIDs and numbers taken from abstracts, is in drafts/A-feasibility.md, drafts/B-surgeon-intervention.md and drafts/C-pathway-and-tte.md, together with files 01 and 02. In total, 220 PMIDs were checked against PubMed; all exist, and a random sample of 15 titles matched the claims made about them. Framing: D54. TTE here means emulation on any observational data: claims, EHR or clinical registry (VQI; VQI linked to Medicare, VISION).

How to read Table 1

• Can TTE solve it? asks whether target trial emulation removes the problem in principle, separately for claims and for EHR/registries, because the answer often depends on the data source.
• Tested in our simulation names the simulated world or analysis that measures it against a known truth. Candidate means not yet simulated.

Draft Table 1. Why surgical RCTs struggle, and whether target trial emulation solves it

#	Challenge	How common / how big	Vascular / claudication example	TTE on claims	TTE on EHR / registry	Tested in our simulation
A. Running the trial
1	Recruitment failure, early stopping, non-publication	Only 14.6% of surgical RCTs finish on time and at full size (Shadbolt 2023). 21% stopped early, mostly for recruitment (Chapman 2014). In vascular surgery, 22% are discontinued and 80% of those are never published (Binko 2023).	CLEVER planned 252 and randomized 111. SUPER planned 400, randomized 240, and stopped early. OBACT randomized 56 of 434.	Yes: no recruitment step	Yes, but registries hold only treated patients, so a no-PVI arm needs linkage	Realistic trial (n = 240) vs TTE on the full population
2	Equipoise is absent, and surgeons have strong preferences	Vascular surgeons' equipoise is often absent, and preference strength predicts refusal to randomize (Young 2004). Clinicians' "hunches" undermine trials (Donovan 2014).	Physician early-PVI rates range from 0% to 58% (Hicks 2020)	Partly: preference becomes the source of treatment variation, and an unmeasured confounder if it tracks prognosis	Partly: drivers such as Rutherford class and ABI are recorded	Practice-style worlds (W2, W3)
3	Who consents is selective	Refusal >50% in most preference trials, related to age, sex and education. Outcomes rarely differ (Wasmann 2019; King 2005; Vist 2005). Preference may modify effects, ES about 0.16 (PCRG 2008).	Whyman: 62 of 600 screened claudicants randomized	Yes for selection (no consent step); patient preference is still unrecorded	Same	Candidate: make consent in the simulated trial depend on patient characteristics (currently random)
4	Too small to detect hard outcomes	Vascular RCT conclusions hinge on 3–7 events (Javidan 2024; Li 2023). Median negative surgical RCT has about 80 patients (Ahmed Ali 2018).	Procedure-related amputation within 1 year after revascularization for claudication is 0.2% (Djerf 2020)	Yes: tens of thousands of patients. A precise estimate can still be biased.	Partly: smaller samples	Realistic trial: power 25% for a true 5-point harm. TTE spread is 1.5–2.8 points.
5	Short follow-up and soft endpoints	Claudication trials use 541 different outcomes and few PROs (Shwan 2024). 88% of device endpoints are surrogates (Dhruva 2009). Benefit fades by 2–5 years (IRONIC, ERASE).	No claudication RCT was powered for CLTI, amputation or death. Only the Swedish SFA trial reports them by arm (about 2% amputation per arm at 5 years, n = 100).	Yes for hard long-term outcomes; no for walking distance and QoL	Partly: walking distance, ABI and Rutherford class recorded at visits, but follow-up is short and fragmented. VQI–Medicare linkage reaches 74–82% (Hoel 2017).	Benefit-then-harm scenario (benefit at 2 years, harm at 5). Candidate: add a 12-month trial analysis.
6	Cost, funding, industry, ethics of withholding or sham	IRONIC revascularization cost about 2× exercise. Trials save money when they use routine data (Griessbach 2024).	No sham-controlled PVI trial exists; ORBITA is the nearest analogue	Yes for cost and withholding; no for placebo questions	Same; symptom outcomes are prone to placebo effects	Not testable (Discussion)
B. The surgeon and the procedure
7	Learning curve	Plateau at about 250 cases (Vickers 2007). 1 of 247 surgical RCTs analysed learning effects (Conroy 2019).	In Medicare claims, carotid stenting by low-experience operators had aOR 1.7 (Nallamothu 2011)	Partly: cumulative operator volume is computable	Partly: plus technical success	Candidate: new learning-curve world
8	Surgeon skill varies	Video-rated skill: bottom vs top quartile complications 14.5% vs 5.2% (Birkmeyer 2013). Skill explains 25.8% of complication variation (Stulberg).	PVI operator volume and MALE, OR 0.73 (Chunawala)	Partly: proxies only (volume)	Partly	Surgeon-skill worlds (W1, W1 ρ = 0)
9	Trials recruit selected surgeons and centres	CEA mortality 1.4% at trial hospitals vs up to 2.5% elsewhere (Wennberg 1998). Trial patients are younger than registry patients (Cutteridge 2025).	Carotid and EVAR trials vs practice	Yes: all operators are included	Partly: centres choose to join registries	Realistic trial (selected surgeons) vs ideal trial vs TTE
10	Procedure not standardized; device heterogeneity	29.4% of trial interventions standardized (Blencowe)	Tibial PVI rate varies by physician, IQR 9–38% (Bose 2023)	Partly: modality and territory only	Mostly: lesion and device data	Not simulated (Discussion: "versions of treatment")
11	Outcomes cluster by surgeon	4% of surgical RCTs adjusted for provider (Biau 2008). Ignoring clustering overstates precision (Lee & Thompson 2005).	Hundreds of PVI operators per registry	Yes for SEs; confounding by physician remains	Yes	Physician variants V1–V4 in every world
12	No blinding; outcome detection bias	Non-blinded assessors inflate subjective effects by 36–68%; objective outcomes are not affected (Hróbjartsson 2012; Wood 2008)	Claims codes separate claudication from CLTI with 81% agreement against VQI (Bose 2025)	No blinding, but amputation and death are objective. CLTI coding is at risk.	No: clinician-reported	Style affects CLTI coding (W3)
C. The treatment pathway
13	Crossover: "conservative" becomes delayed surgery	27–43% crossover within 1–2 years across spine, knee and appendicitis trials, up to two-thirds by 10 years in hernia trials. Effects shrink toward the null (Gaudino 2019); STICH ITT was not significant at 5 years.	SUPER 29% at 1 year; Swedish SFA 25% by 5 years; ERASE later revascularization HR 2.50	Yes: explicit strategies with CCW and sequential trials, if the triggers (worsening) are measured. Claims lack symptoms.	Yes, better: symptoms partly recorded	Deferral world (W4)
14	Patients refuse or don't adhere	ACST: only 50% were operated on by 1 month. SET: 24.2% recruited, US completion 9.8–49.9%.	SUPER SET compliance 50%	Partly: refusal of an offered operation is invisible	Partly: sometimes noted	Trial arm A has 10% refusal
15	Co-interventions and usual care differ	Usual care poorly reported (TIDieR median 1/12). Only 38.1% of VQI claudicants on complete OMT before revascularization.	"Conservative" = advice vs SET + BMT vs cilostazol across trials	Partly: drugs and Medicare SET claims are visible; home exercise and smoking are not	Partly: VQI has OMT only at the procedure	Candidate: unmeasured supervised exercise (supplement)
16	The comparator is ill-defined	Most "conservative" arms are really "surgery if needed". CCW and sequential trials give different numbers (Garcia-Albeniz 2026).	"No PVI for 6 months" vs "PVI only for CLTI" vs "never PVI"	Yes, TTE's main gain: several named strategies from one dataset	Same	Aim 1: A vs B1, A vs B2, sequential trials, landmark
D. The price TTE pays
17	Confounding by indication (what randomization fixes)	TTE agrees with RCTs when the emulation is close (r = 0.93, RCT-DUPLICATE; 0.83, Wang 2026), poorly otherwise (0.53). Only 30.8% of published TTEs address unmeasured confounding (Ren 2026). 19.5% of CCW studies still contain immortal time (Manzanilla 2026).	No claudication or PVI TTE found in PubMed	No: severity poorly captured	Partly: ABI and Rutherford class recorded	Unmeasured-severity world (W5) and realistic world (WR). Candidate: EHR-like world with noisy severity.
18	Misclassified eligibility	About half of registry "claudication" amputations were in misclassified CLTI patients (Djerf 2020)	Claudication vs CLTI codes 81% agreement (Bose 2025)	Risk	Risk	Not simulated (out of scope, D38)

What the review means for the simulation

Already covered (no change needed). Rows 1, 2, 4, 8, 9, 11, 13, 16 and 17 map onto existing worlds or analyses, and rows 12 and 14 are partly covered.

Candidate additions, in order of value for cost:

Candidate	Table 1 row	What it adds	Cost	Recommendation
EHR/registry-like world: severity measured with noise (e.g., ABI and symptom class) instead of exact or only visit counts	17	Answers "does better data close the gap?", which is the claims-vs-EHR question	Parameters only; same DGP as W0, about 1–1.5 h of running	Add
Non-random consent in the simulated trial: consent depends on age and frailty, moderately (Wasmann 2019, King 2005)	3	More realistic trial; the trial's own truth must then be weighted by consent probability	Small code change in rct.R plus RCT truths	Add
12-month trial analysis in the benefit-then-harm scenario	5	Shows a short trial reaching the opposite conclusion from 5-year TTE	Small: ITT at 12 and 24 months plus truths at those horizons	Add
Learning-curve world: skill rises with each operator's cumulative volume	7	TTE-specific issues: interference, delayed PVIs done by more experienced operators, physician confounding that changes over time	Needs calendar-time simulation across patients; a real build (about a day)	Later / supplement
Unmeasured supervised exercise	15	Co-intervention that differs by arm	One more confounder	Supplement (already planned)

Points for the Introduction and Discussion

1. No PubMed hit for a claudication or PVI target trial emulation, no systematic review of TTE in surgery or vascular surgery, and only scattered benchmarks of TTE against procedural RCTs. TTE guides for surgeons exist (e.g., JBJS Rev 2026), but no known-truth tests.
2. No data source wins everywhere. Claims give long, complete follow-up of hard outcomes but miss severity and symptoms. Registries and EHR record severity but have short, fragmented follow-up and no untreated arm unless linked. Linkage (VISION) is incomplete.
3. TTE complements trials rather than replacing them. It covers long-term hard outcomes trials cannot afford, while trials remain needed for walking distance and quality of life.

Open items

• EXITPAD could not be found in PubMed; it is left out of the trial table.
• Some trial details are not in the abstracts and need full texts: planned n for MIMIC, IRONIC and ERASE; crossover in IRONIC, CLEVER and MIMIC; ERASE's long-term CLI result.
• Authors' own papers. Bose 2023 (36058433) and Bose 2025 (40439664, plus a second 2025 paper) are cited neutrally, for practice variation and code validation. The authors should decide whether to keep them.