tte-surgery/docs/literature/03-rct-challenges-and-tte.md

# Literature review 3: challenges of surgical RCTs, and whether TTE solves them

*Merged from three parallel searches (2026-10-28). The evidence, with verified PMIDs and numbers taken from abstracts, is in `drafts/A-feasibility.md`, `drafts/B-surgeon-intervention.md` and `drafts/C-pathway-and-tte.md`, together with files 01 and 02. In total, 220 PMIDs were checked against PubMed; all exist, and a random sample of 15 titles matched the claims made about them. Framing: D54. TTE here means emulation on any observational data: claims, EHR or clinical registry (VQI; VQI linked to Medicare, VISION).*

## How to read Table 1

- **Can TTE solve it?** asks whether target trial emulation removes the problem *in principle*, separately for claims and for EHR/registries, because the answer often depends on the data source.
- **Tested in our simulation** names the simulated world or analysis that measures it against a known truth. *Candidate* means not yet simulated.

## Draft Table 1. Why surgical RCTs struggle, and whether target trial emulation solves it

| # | Challenge | How common / how big | Vascular / claudication example | TTE on claims | TTE on EHR / registry | Tested in our simulation |
|---|---|---|---|---|---|---|
| **A. Running the trial** | | | | | | |
| 1 | Recruitment failure, early stopping, non-publication | Only 14.6% of surgical RCTs finish on time and at full size (Shadbolt 2023). 21% stopped early, mostly for recruitment (Chapman 2014). In vascular surgery, 22% are discontinued and 80% of those are never published (Binko 2023). | CLEVER planned 252 and randomized 111. SUPER planned 400, randomized 240, and stopped early. OBACT randomized 56 of 434. | **Yes**: no recruitment step | **Yes**, but registries hold only treated patients, so a no-PVI arm needs linkage | Realistic trial (n = 240) vs TTE on the full population |
| 2 | Equipoise is absent, and surgeons have strong preferences | Vascular surgeons' equipoise is often absent, and preference strength predicts refusal to randomize (Young 2004). Clinicians' "hunches" undermine trials (Donovan 2014). | Physician early-PVI rates range from 0% to 58% (Hicks 2020) | **Partly**: preference becomes the source of treatment variation, and an unmeasured confounder if it tracks prognosis | **Partly**: drivers such as Rutherford class and ABI are recorded | Practice-style worlds (W2, W3) |
| 3 | Who consents is selective | Refusal >50% in most preference trials, related to age, sex and education. Outcomes rarely differ (Wasmann 2019; King 2005; Vist 2005). Preference may modify effects, ES about 0.16 (PCRG 2008). | Whyman: 62 of 600 screened claudicants randomized | **Yes for selection** (no consent step); patient preference is still unrecorded | Same | *Candidate*: make consent in the simulated trial depend on patient characteristics (currently random) |
| 4 | Too small to detect hard outcomes | Vascular RCT conclusions hinge on 3–7 events (Javidan 2024; Li 2023). Median negative surgical RCT has about 80 patients (Ahmed Ali 2018). | Procedure-related amputation within 1 year after revascularization for claudication is 0.2% (Djerf 2020) | **Yes**: tens of thousands of patients. A precise estimate can still be biased. | **Partly**: smaller samples | Realistic trial: power 25% for a true 5-point harm. TTE spread is 1.5–2.8 points. |
| 5 | Short follow-up and soft endpoints | Claudication trials use 541 different outcomes and few PROs (Shwan 2024). 88% of device endpoints are surrogates (Dhruva 2009). Benefit fades by 2–5 years (IRONIC, ERASE). | No claudication RCT was powered for CLTI, amputation or death. Only the Swedish SFA trial reports them by arm (about 2% amputation per arm at 5 years, n = 100). | **Yes for hard long-term outcomes; no** for walking distance and QoL | **Partly**: walking distance, ABI and Rutherford class recorded at visits, but follow-up is short and fragmented. VQI–Medicare linkage reaches 74–82% (Hoel 2017). | Benefit-then-harm scenario (benefit at 2 years, harm at 5). *Candidate*: add a 12-month trial analysis. |
| 6 | Cost, funding, industry, ethics of withholding or sham | IRONIC revascularization cost about 2× exercise. Trials save money when they use routine data (Griessbach 2024). | No sham-controlled PVI trial exists; ORBITA is the nearest analogue | **Yes** for cost and withholding; **no** for placebo questions | Same; symptom outcomes are prone to placebo effects | Not testable (Discussion) |
| **B. The surgeon and the procedure** | | | | | | |
| 7 | Learning curve | Plateau at about 250 cases (Vickers 2007). 1 of 247 surgical RCTs analysed learning effects (Conroy 2019). | In Medicare claims, carotid stenting by low-experience operators had aOR 1.7 (Nallamothu 2011) | **Partly**: cumulative operator volume is computable | **Partly**: plus technical success | *Candidate*: new learning-curve world |
| 8 | Surgeon skill varies | Video-rated skill: bottom vs top quartile complications 14.5% vs 5.2% (Birkmeyer 2013). Skill explains 25.8% of complication variation (Stulberg). | PVI operator volume and MALE, OR 0.73 (Chunawala) | **Partly**: proxies only (volume) | **Partly** | Surgeon-skill worlds (W1, W1 ρ = 0) |
| 9 | Trials recruit selected surgeons and centres | CEA mortality 1.4% at trial hospitals vs up to 2.5% elsewhere (Wennberg 1998). Trial patients are younger than registry patients (Cutteridge 2025). | Carotid and EVAR trials vs practice | **Yes**: all operators are included | **Partly**: centres choose to join registries | Realistic trial (selected surgeons) vs ideal trial vs TTE |
| 10 | Procedure not standardized; device heterogeneity | 29.4% of trial interventions standardized (Blencowe) | Tibial PVI rate varies by physician, IQR 9–38% (Bose 2023) | **Partly**: modality and territory only | **Mostly**: lesion and device data | Not simulated (Discussion: "versions of treatment") |
| 11 | Outcomes cluster by surgeon | 4% of surgical RCTs adjusted for provider (Biau 2008). Ignoring clustering overstates precision (Lee & Thompson 2005). | Hundreds of PVI operators per registry | **Yes** for SEs; confounding by physician remains | **Yes** | Physician variants V1–V4 in every world |
| 12 | No blinding; outcome detection bias | Non-blinded assessors inflate subjective effects by 36–68%; objective outcomes are not affected (Hróbjartsson 2012; Wood 2008) | Claims codes separate claudication from CLTI with 81% agreement against VQI (Bose 2025) | **No blinding**, but amputation and death are objective. CLTI coding is at risk. | **No**: clinician-reported | Style affects CLTI coding (W3) |
| **C. The treatment pathway** | | | | | | |
| 13 | Crossover: "conservative" becomes delayed surgery | 27–43% crossover within 1–2 years across spine, knee and appendicitis trials, up to two-thirds by 10 years in hernia trials. Effects shrink toward the null (Gaudino 2019); STICH ITT was not significant at 5 years. | SUPER 29% at 1 year; Swedish SFA 25% by 5 years; ERASE later revascularization HR 2.50 | **Yes**: explicit strategies with CCW and sequential trials, **if** the triggers (worsening) are measured. Claims lack symptoms. | **Yes, better**: symptoms partly recorded | Deferral world (W4) |
| 14 | Patients refuse or don't adhere | ACST: only 50% were operated on by 1 month. SET: 24.2% recruited, US completion 9.8–49.9%. | SUPER SET compliance 50% | **Partly**: refusal of an offered operation is invisible | **Partly**: sometimes noted | Trial arm A has 10% refusal |
| 15 | Co-interventions and usual care differ | Usual care poorly reported (TIDieR median 1/12). Only 38.1% of VQI claudicants on complete OMT before revascularization. | "Conservative" = advice vs SET + BMT vs cilostazol across trials | **Partly**: drugs and Medicare SET claims are visible; home exercise and smoking are not | **Partly**: VQI has OMT only at the procedure | *Candidate*: unmeasured supervised exercise (supplement) |
| 16 | The comparator is ill-defined | Most "conservative" arms are really "surgery if needed". CCW and sequential trials give different numbers (Garcia-Albeniz 2026). | "No PVI for 6 months" vs "PVI only for CLTI" vs "never PVI" | **Yes, TTE's main gain**: several named strategies from one dataset | Same | Aim 1: A vs B1, A vs B2, sequential trials, landmark |
| **D. The price TTE pays** | | | | | | |
| 17 | Confounding by indication (what randomization fixes) | TTE agrees with RCTs when the emulation is close (r = 0.93, RCT-DUPLICATE; 0.83, Wang 2026), poorly otherwise (0.53). Only 30.8% of published TTEs address unmeasured confounding (Ren 2026). 19.5% of CCW studies still contain immortal time (Manzanilla 2026). | No claudication or PVI TTE found in PubMed | **No**: severity poorly captured | **Partly**: ABI and Rutherford class recorded | Unmeasured-severity world (W5) and realistic world (WR). *Candidate*: EHR-like world with noisy severity. |
| 18 | Misclassified eligibility | About half of registry "claudication" amputations were in misclassified CLTI patients (Djerf 2020) | Claudication vs CLTI codes 81% agreement (Bose 2025) | **Risk** | **Risk** | Not simulated (out of scope, D38) |

## What the review means for the simulation

**Already covered (no change needed).** Rows 1, 2, 4, 8, 9, 11, 13, 16 and 17 map onto existing worlds or analyses, and rows 12 and 14 are partly covered.

**Candidate additions, in order of value for cost:**

| Candidate | Table 1 row | What it adds | Cost | Recommendation |
|---|---|---|---|---|
| **EHR/registry-like world**: severity measured with noise (e.g., ABI and symptom class) instead of exact or only visit counts | 17 | Answers "does better data close the gap?", which is the claims-vs-EHR question | Parameters only; same DGP as W0, about 1–1.5 h of running | **Add** |
| **Non-random consent in the simulated trial**: consent depends on age and frailty, moderately (Wasmann 2019, King 2005) | 3 | More realistic trial; the trial's own truth must then be weighted by consent probability | Small code change in `rct.R` plus RCT truths | **Add** |
| **12-month trial analysis** in the benefit-then-harm scenario | 5 | Shows a short trial reaching the opposite conclusion from 5-year TTE | Small: ITT at 12 and 24 months plus truths at those horizons | **Add** |
| **Learning-curve world**: skill rises with each operator's cumulative volume | 7 | TTE-specific issues: interference, delayed PVIs done by more experienced operators, physician confounding that changes over time | Needs calendar-time simulation across patients; a real build (about a day) | **Later / supplement** |
| **Unmeasured supervised exercise** | 15 | Co-intervention that differs by arm | One more confounder | Supplement (already planned) |

## Points for the Introduction and Discussion

1. **No PubMed hit for a claudication or PVI target trial emulation**, no systematic review of TTE in surgery or vascular surgery, and only scattered benchmarks of TTE against procedural RCTs. TTE guides for surgeons exist (e.g., *JBJS Rev* 2026), but no known-truth tests.
2. **No data source wins everywhere.** Claims give long, complete follow-up of hard outcomes but miss severity and symptoms. Registries and EHR record severity but have short, fragmented follow-up and no untreated arm unless linked. Linkage (VISION) is incomplete.
3. **TTE complements trials rather than replacing them.** It covers long-term hard outcomes trials cannot afford, while trials remain needed for walking distance and quality of life.

## Open items

- EXITPAD could not be found in PubMed; it is left out of the trial table.
- Some trial details are not in the abstracts and need full texts: planned n for MIMIC, IRONIC and ERASE; crossover in IRONIC, CLEVER and MIMIC; ERASE's long-term CLI result.
- **Authors' own papers.** Bose 2023 (36058433) and Bose 2025 (40439664, plus a second 2025 paper) are cited neutrally, for practice variation and code validation. The authors should decide whether to keep them.