255 lines
32 KiB
Markdown
255 lines
32 KiB
Markdown
# Draft A: trial feasibility and design challenges in surgical RCTs
|
||
|
||
Draft input for Table 1. It covers six challenges: recruitment, equipoise, consent selection, sample size, endpoints and follow-up, and cost, funding and ethics.
|
||
|
||
**Method.** Every PMID below was checked with NCBI esummary, and the abstract was read with efetch, on 2026-10-28. Numbers come from the PubMed abstracts only; no full texts were read. Papers already listed in `01-surgical-rct-limitations.md` or `02-claudication-evidence-and-methods.md` are not listed again. They are cited by PMID where they help the argument and are marked "(file 01)" or "(file 02)".
|
||
|
||
**Data sources for TTE.** We assume three kinds of data, and they differ in what they can do:
|
||
- **Claims** (Medicare fee-for-service): population-based, with long and nearly complete follow-up for death, procedures, amputation and diagnoses. They contain no walking distance, ABI, Rutherford class, or patient-reported outcomes (PROs).
|
||
- **EHR and clinical registries** (VQI): severity at the time of the procedure (Rutherford class, ABI, anatomy) and some PROs. Follow-up is short or fragmented, and VQI captures patients only when they have a procedure, so patients managed without one are mostly missing.
|
||
- **Linked registry–claims data** (VQI–Medicare, VISION): registry severity plus claims follow-up. Linkage is incomplete, and the cohort is still defined by having a procedure.
|
||
|
||
---
|
||
|
||
## 1. Recruitment, slow accrual, early discontinuation, non-publication
|
||
|
||
**What it is.** Surgical trials often fail to recruit the planned number of patients, run years late, or stop early. Trials that stop early are rarely published, and so are many completed trials. The evidence base is therefore smaller and more biased than the registered trial activity suggests.
|
||
|
||
**How common / how big.**
|
||
- Completion on time and at full size is rare. Of 2,542 surgical RCTs registered in 2010–2014, only **14.6%** finished both on time and at full enrollment. **21.5%** were discontinued, and when they stopped they were a median **92.7%** short of their enrollment target (36648945).
|
||
- Surgical trials are discontinued more often than nonsurgical ones. Among 88,498 US trials, discontinuation was **15.97% vs 11.07%**, and poor recruitment was cited more often (**44.65% vs 34.74%**) (29804847).
|
||
- Industry funding predicted discontinuation (OR 1.25 in surgical trials) (29804847).
|
||
- These figures build on Chapman 2014 (25491195), Rosenthal 2015 (24979608) and Kasenda 2014 (24618966) (file 01).
|
||
|
||
**Vascular / claudication example.**
|
||
- Vascular surgery RCTs: of 108 registered in 2010–2019, **22.2%** were discontinued, most often for poor enrollment (**45.8%**). Discontinued trials reached only **28.4%** of planned enrollment, and **80%** of them were never published. Of completed trials, **25%** were unpublished; industry funding cut the odds of publication (OR 0.18). In total, **4,788 enrollees** have no public results (37311508).
|
||
- PAD exercise trials: one recruitment effort spent $1,330 on community outreach and got two referrals and no enrollments. Health-system referral produced 44 referrals and one enrollment (30139448).
|
||
- Cardiac surgery is a neighbouring field: **22.6%** of trials were terminated early, and aortic surgery trials had higher odds of termination (OR 2.86) (39133169).
|
||
- Under-representation of women and minorities is a related problem. Across 110 North American PAD trials (57,451 patients), **28%** of patients were women and **5.0%** were Black (42051894).
|
||
- BASIL-3 reaching 481 of 861 planned patients is noted in file 01.
|
||
|
||
| Paper | PMID | Takeaway |
|
||
|---|---|---|
|
||
| Shadbolt C et al. 2023, *JAMA Netw Open* | 36648945 | 2,542 surgical RCTs. 14.6% finished on time and at full size, 20.4% on time, and 45.9% met their enrollment target. Median delay was 12.2 months (66.7% longer than planned). 21.5% were discontinued, with a median shortfall of 92.7%. |
|
||
| Mouw TJ et al. 2018, *J Surg Res* | 29804847 | 5,779 surgical vs 82,719 nonsurgical US trials. Discontinued: 15.97% vs 11.07%. Poor recruitment was cited in 44.65% vs 34.74%. Industry funding predicted discontinuation. |
|
||
| **Binko MA et al. 2023, *Ann Vasc Surg*** | 37311508 | Vascular surgery RCTs: 22.2% discontinued, 45.8% of them for poor enrollment. Of trials stopped after enrollment began, 80% were unpublished. Of completed trials, 25% were unpublished (industry funding OR 0.18 for publication). |
|
||
| Cancelli G et al. 2024, *Eur J Cardiothorac Surg* | 39133169 | 746 cardiac surgery trials: 22.6% terminated early, 29.6% of them for slow recruitment. Odds of termination were higher for valve (OR 4.30), LVAD (OR 3.65) and aortic (OR 2.86) trials. |
|
||
| Caldieraro-Bentley AJ et al. 2018, *J Vasc Nurs* | 30139448 | PAD exercise trials. Indirect recruitment ($1,330): 2 referrals, 0 enrollments. Direct recruitment: 44 referrals, 1 enrollment. Barriers were comorbidity, transport, and strict eligibility criteria. |
|
||
| Shah T et al. 2026, *JSCAI* | 42051894 | 110 North American PAD trials (2000–2023): 28% women, 5.0% Black, 1.3% Hispanic. 80% were industry-funded. |
|
||
|
||
**Can TTE solve it? Yes for feasibility; partly for representativeness, depending on the data source.**
|
||
- TTE has no recruitment, consent or discontinuation step, so all eligible patients in the data enter the emulation.
|
||
- **Claims:** Medicare covers nearly every older US claudicant, including women, Black patients and rural patients whom trials under-enroll. Patients under 65 and those in Medicare Advantage are missing.
|
||
- **Registries (VQI):** these include only patients who had a procedure, at participating centres. A registry cannot supply patients managed without a procedure (the "delayed / never" arm) unless it is linked to claims or an EHR.
|
||
- Non-publication is a separate problem. TTE analyses can also go unpublished selectively, so a protocol should be registered in advance (TARGET, 40899949, file 01).
|
||
|
||
**Testable in a known-truth simulation?** Partly. Simulate a trial that recruits a small, under-powered and selected subset (see §3) and compare it with a TTE run on the full simulated population. The selective-publication part is not testable.
|
||
|
||
---
|
||
|
||
## 2. Equipoise: surgeon and patient treatment preferences
|
||
|
||
**What it is.**
|
||
- Randomization is ethical only under equipoise, which is genuine uncertainty about which option is better. Freedman's "clinical equipoise" places that uncertainty in the expert community rather than in each individual clinician.
|
||
- Surgeons often hold strong individual beliefs and tolerate uncertainty poorly, so they may not offer the trial or may selectively enrol patients.
|
||
- Patients also bring strong preferences, especially when one arm is "an operation" and the other is "no operation".
|
||
|
||
**How common / how big.**
|
||
- Surgeons were significantly more intolerant of uncertainty than physicians (P = 0.007), but their attitudes to RCTs did not differ (16078288).
|
||
- 62.6% of Australian colorectal surgeons named difficulty convincing patients to accept random allocation as a barrier (11598468).
|
||
- In a US national survey, there was consensus (≥60% agreement) on whether to operate in 19 of 30 cases. Variation was largest when the modal answer was "do not operate" or "equipoise" (38328985).
|
||
- In spine surgery, surgeons judged 12 of 20 cases eligible for randomization (17304133).
|
||
- File 01 covers Donovan 2014 (24811157) on clinicians' "hunches" and Rooshenas 2016 (27755555) on how equipoise is presented in consultations.
|
||
|
||
**Vascular / claudication example.**
|
||
- In a survey of vascular surgeons in Australia and New Zealand (77% response), **24%** were in individual equipoise on endarterectomy vs stenting for high-risk carotid patients. **89%** favoured endarterectomy for average-risk patients, so there was no community equipoise. Strength of preference predicted unwillingness to join 4 of 6 hypothetical trials (15523403).
|
||
- Patients with severe claudication were **risk-seeking** about invasive treatment, and patients who could walk only short distances accepted more risk (18455642).
|
||
- In Medicare, the physician-level rate of tibial PVI for claudication varied widely: median **20.0%**, IQR **9.1–37.5%** (36058433). This is practice variation that no trial would capture.
|
||
|
||
| Paper | PMID | Takeaway |
|
||
|---|---|---|
|
||
| **Freedman B 1987, *NEJM*** | 3600702 | Defines "clinical equipoise": uncertainty in the expert community. Requiring individual equipoise "presents nearly insuperable obstacles" to starting a trial and can cause recruitment failure. |
|
||
| McCulloch P et al. 2005, *Br J Surg* | 16078288 | 2,000 consultants surveyed (36.5% responded). Surgeons were more intolerant of uncertainty (P = 0.007) but not more hostile to RCTs. Intolerance may block decisions to enrol individual patients. |
|
||
| **Young J et al. 2004, *Surgery*** | 15523403 | Vascular surgeons: 24% in equipoise on carotid stenting in high-risk patients, 89% preferred endarterectomy in average-risk patients, and 88% favoured a minimally invasive approach for high-risk AAA. Preference strength predicted unwillingness to randomize. |
|
||
| Gattellari M et al. 2001, *Dis Colon Rectum* | 11598468 | 89% response. 66.7% said RCTs should be the design of choice. Barriers: funding (74.4%), convincing patients to accept random allocation (62.6%), lack of wider surgical support (55.9%). |
|
||
| Zychowski KL et al. 2024, *Ann Surg* | 38328985 | 580 US surgeons. Consensus on 19 of 30 cases, with most variation where the mode was against surgery or at equipoise. Surgeons often "let patients decide". |
|
||
| Ghogawala Z et al. 2007, *Spine* | 17304133 | Spine surgeons split 51% / 38% / 11% between approaches. Only 12 of 20 real cases were judged eligible for randomization. |
|
||
| Letterstål A et al. 2008, *J Vasc Surg* | 18455642 | 50 patients with severe claudication: EQ-5D 0.51, standard gamble 0.88. Patients were risk-seeking about surgery, more so if their walking distance was short. |
|
||
| Bose S et al. 2023, *J Vasc Surg* | 36058433 | Medicare, 59,930 PVIs for claudication: 27.7% tibial. Physician-level rate median 20.0% (IQR 9.1–37.5%). Practice setting (office-based lab aOR 2.37) and specialty predicted use. |
|
||
|
||
**Can TTE solve it? Partly.**
|
||
- TTE needs no one to be in equipoise. It instead relies on preference-driven variation between physicians and regions, which is exactly what makes trials hard to run. Physician preference (for example, a physician's past PVI rate) can serve as an instrument or as a source of positivity.
|
||
- Patient preference, and the clinical judgement behind a surgeon's choice for a particular patient, are not recorded. If they predict outcomes, they are unmeasured confounders.
|
||
- **EHR and registries** partly measure what drives the choice (Rutherford class, ABI, anatomy), which narrows the problem. **Claims** do not.
|
||
- Wherever community consensus is near-total, TTE suffers positivity violations, just as a trial would struggle to recruit.
|
||
|
||
**Testable in a known-truth simulation? Yes.** Generate a latent physician "intervention propensity" (already in the data-generating process, or DGP) and a latent patient preference that affects both treatment and outcome. Vary how strongly each confounds the outcome, and check how much TTE bias each produces.
|
||
|
||
---
|
||
|
||
## 3. Patient preference and consent selection: who agrees to be randomized
|
||
|
||
**What it is.** Eligible patients who decline randomization usually do so because they prefer one treatment. Trial participants can therefore differ from the eligible population, which limits external validity. If preference also changes how patients respond, it can affect the outcome. Comprehensive-cohort and partially randomized preference designs follow patients who decline, so the two groups can be compared directly.
|
||
|
||
**How common / how big.**
|
||
- Partially randomized preference trials (44 trials, 24,873 patients): refusal of randomization was **over 50% in 26 of 44 trials** (range 19–99%). Decliners differed by education, sex, age, race and prior experience of one treatment. Primary outcomes were similar in the randomized and preference cohorts (mean difference 0.093, 95% CI −0.178 to 0.364) (31619428).
|
||
- King 2005 (32 comprehensive cohorts and two-stage trials): preferences led "a substantial proportion" to refuse randomization, but there was little evidence of bias in who agreed, or in outcomes (15741531).
|
||
- Musculoskeletal trials, patient-level data (n = 1,594): patients randomized to their preferred treatment did better than indifferent patients (effect size **0.162**, 95% CI 0.011–0.314) (18977792).
|
||
- Trial participants vs comparable non-participants (31,140 vs 20,380 patients): no overall difference in outcomes (SMD 0.01), but heterogeneity was high (I² = 89%) for dichotomous outcomes (15905256).
|
||
|
||
**Surgical examples (no vascular example with a non-participant cohort was found).**
|
||
- SPORT disc herniation: the 743 patients who declined randomization formed an observational cohort. Their self-selected surgery-vs-nonoperative difference was large (bodily pain 14.8 points at 3 months), whereas the randomized ITT difference was not significant (17119141; ITT trial 17119140, file 01).
|
||
- PREOPANC (pancreatic cancer): only **33%** of 455 eligible patients were randomized, and 55% of the rest refused. Survival (15.2 vs 15.5 months) and resection rates were the same in the two groups (33273349).
|
||
- ProtecT (prostate cancer): men who chose their treatment were similar to randomized men, apart from occupation (29288137).
|
||
- Meniscal tear: willingness to join a planned surgery-vs-nonoperative RCT fell from **62% to 46%** after a course of physiotherapy (41367623).
|
||
|
||
| Paper | PMID | Takeaway |
|
||
|---|---|---|
|
||
| **King M et al. 2005, *JAMA*** | 15741531 | 32 preference-design studies. Preference drives refusal, but there was little evidence that it biases who enrols or the trial's internal validity. |
|
||
| **Preference Collaborative Review Group 2008, *BMJ*** | 18977792 | Patient-level data, n = 1,594. Randomized to preferred treatment vs indifferent: ES 0.162 (0.011–0.314). Allocated to an unwanted treatment: OR 1.70 for loss to first follow-up compared with indifferent patients. |
|
||
| **Wasmann KA et al. 2019, *BMJ Open*** | 31619428 | 44 partially randomized preference trials (n = 24,873). Refusal over 50% in 26 trials. Decliners differed by education, sex, age, race and prior treatment. Outcomes similar between cohorts. Loss to follow-up and crossover were higher in the randomized cohort. |
|
||
| Vist GE et al. 2005, *BMJ* | 15905256 | Trial participants vs non-participants: no consistent effect of participation (SMD 0.01), but large heterogeneity (I² = 89%). |
|
||
| **Weinstein JN et al. 2006, *JAMA* (SPORT observational cohort)** | 17119141 | Patients who declined randomization and chose surgery improved more (treatment effect 14.8 to 15.4 points at 3 months). The authors warn that this comparison is confounded. |
|
||
| Versteijne E et al. 2022, *Ann Surg* (PREOPANC) | 33273349 | 33% of eligible patients randomized. Non-randomized patients had the same survival and resection rates. |
|
||
| Donovan JL et al. 2018, *J Clin Epidemiol* (ProtecT) | 29288137 | Extended comprehensive cohort: 1,643 randomized, 997 chose treatment, and 557 were excluded. Those who chose were similar to the randomized men; those excluded were sicker. |
|
||
| Fox KB et al. 2025, *Osteoarthr Cartil Open* | 41367623 | Advance preference assessment: willingness to enrol in a surgery-vs-nonoperative RCT fell from 62% to 46%. The strongest preference was to avoid surgery (47%). |
|
||
|
||
**Can TTE solve it? Mostly for selection into the study; no for preference as a confounder.**
|
||
- TTE does not need consent, so it estimates the effect in everyone eligible rather than in those willing to be randomized.
|
||
- The same preference that drives refusal in a trial drives treatment choice in observational data, as SPORT's observational cohort shows. Neither claims nor EHR record patient preference.
|
||
- If preference also modifies the treatment effect, as PCRG 2008 suggests, the trial and the TTE estimate different estimands even when both are unbiased.
|
||
|
||
**Testable in a known-truth simulation? Yes, and this is directly relevant to our design.** The evidence supports making consent to the simulated trial **non-random but only moderately selective**:
|
||
- consent depends on covariates (age, sex, prior treatment) and on a latent preference;
|
||
- refusal rates are well over 50% (Wasmann 2019);
|
||
- consent has only a small effect on outcomes (King 2005, Vist 2005);
|
||
- add an optional scenario with preference × treatment effect modification of about 0.15 SD (PCRG 2008).
|
||
|
||
The trial's ITT estimate for consenters can then be compared with the TTE estimate for the whole population, against the known truth for each.
|
||
|
||
---
|
||
|
||
## 4. Sample size, underpowering, imprecision
|
||
|
||
**What it is.** Surgical trials are small because recruitment is hard, funding is limited, and the evidence is spread across many devices and techniques. Small trials miss real effects, over-estimate the effects they do find, and depend on a handful of events. Hard outcomes in claudication, such as amputation and progression to CLTI, are rare, so trials powered for them would need thousands of patients.
|
||
|
||
**How common / how big.**
|
||
- Only **53%** of surgical RCTs in leading journals reported their sample-size calculation adequately, falling to **12%** against the full SPIRIT items (31787318).
|
||
- File 01 already covers Ahmed Ali 2018 (29907196): the median size of negative surgical RCTs is about 76–80.
|
||
|
||
**Vascular / claudication example.**
|
||
- Fragility: in positive open-vs-endovascular RCTs, a median of **3** event changes would reverse significance, and **70%** of outcomes had more patients lost to follow-up than that (37315910). In negative trials, a median of **7** changes would make them significant (7 for PAD), and **47%** had more patients lost than that (36572321).
|
||
- Cochrane (file 02): **10 RCTs, 1,087 participants** in total for endovascular revascularisation in claudication. Evidence was downgraded for "small sample size", and the endovascular vs no-specific-treatment comparison rested on 3 studies with **125 participants** (29518253).
|
||
- Device dispersal: 112 PAD endovascular RCTs tested **228 different devices or techniques** (37678660).
|
||
- Rare hard outcomes: major amputation within 1 year of revascularisation for claudication was **0.2%** (9 of 5,860; Swedvasc) (31866238).
|
||
- Progression to CLTI under conservative management ranged from **1.1% to 36.7%** across studies, with I² = 99%. Rates that uncertain make power calculations unreliable (39694186).
|
||
|
||
| Paper | PMID | Takeaway |
|
||
|---|---|---|
|
||
| **Javidan A et al. 2024, *J Vasc Surg*** | 37315910 | 21 positive open-vs-endovascular RCTs. Median fragility index 3 (3, 20). In 70% of outcomes, loss to follow-up exceeded the fragility index. |
|
||
| **Li A et al. 2023, *J Vasc Surg*** | 36572321 | 49 negative vascular RCTs. Median reverse fragility index 7 (PAD 7). In 47%, loss to follow-up exceeded it. |
|
||
| Speich B et al. 2020, *Surgery* | 31787318 | 85 surgical RCTs (2017). Sample-size reporting adequate in 53% by CONSORT and 12% with SPIRIT items. |
|
||
| Zywicka EM et al. 2024, *EJVES* | 37678660 | 112 PAD endovascular RCTs covering 228 devices or techniques. Interventions were described well enough to replicate in only 21%. |
|
||
| Fakhry F et al. 2018, *Cochrane* (file 02) | 29518253 | 10 RCTs, n = 1,087, downgraded for small size and heterogeneity. Long-term data on walking distance came from only 2–3 studies (103–184 patients). |
|
||
| Djerf H et al. 2020, *EJVES* | 31866238 | Procedure-related major amputation 0.2% at 1 year after revascularisation for claudication. Of 109 registered amputations, 51 were in patients with CLTI misclassified as claudication. |
|
||
| Froud JLJ et al. 2025, *Ann Vasc Surg* | 39694186 | 9 cohorts (n = 4,115): progression to CLTI 1.1–36.7%, I² = 99%. Women made up 22.7% of patients. |
|
||
|
||
**Can TTE solve it? Yes for size; partly for precision on what matters.**
|
||
- **Claims** offer very large samples (tens of thousands of claudicants), which makes rare hard outcomes (amputation, CLTI, death) estimable with narrow confidence intervals.
|
||
- Large samples do not remove bias. A precise but confounded estimate can mislead more than a wide trial estimate does.
|
||
- **Registries** are smaller and centre-limited but record severity. Djerf 2020 shows that registry indication codes can be wrong: about half of the "claudication" amputations were in misclassified CLTI patients. Misclassification of eligibility is therefore a real risk in claims and in registries.
|
||
- Effective sample size can be much smaller than the raw count after cloning, censoring and weighting, especially with extreme weights.
|
||
|
||
**Testable in a known-truth simulation? Yes.** Compare the sampling variability and coverage of a small trial (n ≈ 100–200) with a population-scale TTE across replicates. Show the bias–variance trade-off: unbiased but noisy trial vs precise TTE with possible bias.
|
||
|
||
---
|
||
|
||
## 5. Short follow-up and soft or surrogate endpoints
|
||
|
||
**What it is.**
|
||
- Claudication trials mostly measure treadmill walking distance, ABI, patency or quality of life at 6–12 months. Patients and payers care about durability, reintervention, CLTI, amputation and death over years.
|
||
- Device trials rely heavily on surrogate endpoints such as patency.
|
||
- Outcome definitions vary so much that trials cannot be pooled.
|
||
- Long follow-up raises costs and loss to follow-up, especially for patient-reported outcomes.
|
||
|
||
**How common / how big.**
|
||
- Of 213 primary endpoints in FDA premarket approvals of cardiovascular devices, **88%** were surrogates. Only **27%** of the supporting studies were randomized (20040556).
|
||
- In surgical RCTs, loss to follow-up was far higher when outcomes were patient-reported (RR **14.21**) and when follow-up lasted 3 months or more (OR **10.09**) (38795671).
|
||
|
||
**Vascular / claudication example.**
|
||
- **408 claudication studies** used **541 unique outcomes**. 386 of these were used only once, ABI was the most common, PROs appeared in only **36.2%** of studies, and definitions of MALE and primary patency varied (39485886).
|
||
- A new core outcome set for chronic symptomatic PAD has 11 outcomes, a mix of hard and patient-centred ones: death, MI, major amputation, acute limb ischaemia, pain-free walking distance, quality of life, and others (40992592).
|
||
- Trials in file 02: CLEVER (6-month treadmill, n = 111; 22090168), ERASE (12 months, n = 212; 26547465) and IRONIC (1 year; benefit lost by 5 years; 25095886, 31937137).
|
||
- The paclitaxel mortality signal appeared only once long-term data were pooled (30561254, file 02).
|
||
- SWEDEPAD 2 is the counter-example: a registry-based RCT with **98.3%** follow-up and a median of **7.1 years** for mortality (40902614, file 02).
|
||
|
||
| Paper | PMID | Takeaway |
|
||
|---|---|---|
|
||
| **Shwan A et al. 2024, *BJS Open*** | 39485886 | 408 claudication studies with 541 unique outcomes; 386 were used only once. PROs in 36.2%. MALE and patency defined inconsistently. Only 38.9% of studies included claudicants alone. |
|
||
| Zwetsloot SLM et al. 2026, *EJVES* | 40992592 | Delphi core outcome set for chronic symptomatic PAD: 11 outcomes, including pain-free walking distance, physical activity, quality of life, amputation, and death. |
|
||
| **Dhruva SS et al. 2009, *JAMA*** | 20040556 | 78 cardiovascular device premarket approvals: 27% of studies randomized, 14% blinded, 65% based on a single study, 88% of primary endpoints surrogate. |
|
||
| Walker RJB et al. 2024, *J Surg Res* | 38795671 | 200 surgical RCTs. Loss to follow-up 3.10 per 100 patient-years. PROs RR 14.21; follow-up ≥3 months OR 10.09; standard-of-care assessments RR 0.17. |
|
||
| Hoel AW et al. 2017, *J Vasc Surg* | 28222989 | VQI matched to Medicare: 82.4% of EVAR and 74.4% of open repair patients matched. Agreement was good for mortality (κ 0.64–0.82) but poor for reintervention (κ 0.16–0.26). |
|
||
|
||
**Can TTE solve it? Partly, and the answer depends on the data source.**
|
||
- **Claims:** strong on long, complete follow-up of hard outcomes (death, amputation, reintervention, CLTI diagnosis codes) over 5–10 years. No walking distance, ABI, Rutherford class or quality of life, so the trials' main efficacy endpoints cannot be emulated.
|
||
- **EHR and registries (VQI):** record walking distance, ABI, Rutherford class, and sometimes PROs, so soft and functional endpoints and baseline severity are partly available. Follow-up is short and fragmented, and functional outcomes are measured only at clinical visits, so they are missing not at random.
|
||
- **Linked VQI–Medicare (VISION):** combines registry severity with claims follow-up. Linkage is incomplete (74–82% in Hoel 2017), and reintervention agrees poorly between sources.
|
||
- No source captures the main claudication benefit (walking and quality of life) with long follow-up. TTE complements trials by covering the long-term hard outcomes they cannot afford to measure, rather than replacing them for symptom endpoints.
|
||
|
||
**Testable in a known-truth simulation? Yes, for hard outcomes and time-varying effects.** The DGP already has an early benefit and late harm on CLTI and amputation. Show that a 12-month trial and a long-follow-up TTE reach different conclusions. PRO endpoints could be added as a latent outcome that is observed only at visits, to show missingness in EHR data, but this is optional.
|
||
|
||
---
|
||
|
||
## 6. Cost, funding, industry sponsorship, and ethics (sham surgery, randomizing to no treatment)
|
||
|
||
**What it is.**
|
||
- Surgical RCTs are expensive and are rarely funded by public bodies. Device makers fund many vascular trials, which affects which questions get asked, which endpoints are used, and whether and how results are published.
|
||
- Placebo (sham) procedures are needed to separate the specific effect of a symptom-relieving procedure from the placebo effect, but they raise ethical and recruitment problems.
|
||
- Randomizing patients to "no procedure" can feel unethical to surgeons and patients when practice favours intervention.
|
||
|
||
**How common / how big.**
|
||
- Cost: the median investigator-initiated RCT cost **$645,824**, and **69.7%** exceeded their budget by more than 50%. Using routinely collected data for the primary outcome was associated with lower costs (39307405).
|
||
- Public funding in vascular surgery: only **1%** of US vascular surgeons (41 of 4,037) hold NIH funding, and only **25%** of their projects are clinical or health-services research (37327950).
|
||
- Industry outcomes across fields: industry-sponsored studies more often have favourable efficacy results (RR **1.27**) and favourable conclusions (RR **1.34**), with no difference between drug and device studies (28207928).
|
||
- Industry outcomes in surgery: industry funding was associated with pro-industry significant results in surgical trials (OR **8.0**, 95% CI 1.1–53.2) (14970094).
|
||
- Device publication: only **49%** of 177 premarket approval studies for cardiovascular devices were published, and when published, primary endpoints were often relabelled or missing (26063311).
|
||
- Sham surgery: placebo-controlled surgical trials recruit as well as non-operative-control trials, but have about twice the attrition (IRR **1.8**) (38637129).
|
||
- Willingness for sham trials: **31%** of surgeons and **40%** of patients would not take part in any placebo-controlled knee trial design offered (39615522).
|
||
|
||
**Vascular / claudication example.**
|
||
- Funding: **80%** of PAD trials were industry-funded (42051894, §1). Industry-sponsored vascular RCTs rose from **6.7% to 28%** over a decade (26298221). Industry funding predicted non-publication of completed vascular RCTs (OR 0.18; 37311508, §1). Jones 2026 found 39% industry-funded (42526742, file 01).
|
||
- Sham procedures: ORBITA showed that a placebo-controlled trial of a symptom-relieving revascularization is possible. PCI did not improve exercise time beyond placebo (**16.6 s**, 95% CI −8.9 to 42.0) (29103656). No sham-controlled trial of PVI for claudication was found; this is an inference, since the search was not exhaustive.
|
||
- Ethics of "no procedure": in claudication, randomizing to exercise alone is ethically easy, because exercise is guideline first-line therapy.
|
||
|
||
| Paper | PMID | Takeaway |
|
||
|---|---|---|
|
||
| Griessbach A et al. 2024, *J Clin Epidemiol* | 39307405 | 93 investigator-initiated RCTs. Median cost $645,824. 69.7% overran their budget by more than 50%. Routinely collected outcome data were linked to lower cost. |
|
||
| **Mirzaie AA et al. 2023, *J Vasc Surg*** | 37327950 | 1% of US vascular surgeons hold NIH funding (41 of 4,037). 75% of projects are basic or translational. |
|
||
| **Lundh A et al. 2017, *Cochrane*** | 28207928 | 75 studies. Industry sponsorship: favourable results RR 1.27, favourable conclusions RR 1.34, same for devices as for drugs. Bias not explained by standard risk-of-bias tools. |
|
||
| Bhandari M et al. 2004, *CMAJ* | 14970094 | 332 RCTs. Industry funding linked to pro-industry results (adjusted OR 1.8). In surgical trials, OR 8.0 (1.1–53.2). |
|
||
| Dhruva 2009 (see §5); Chang L et al. 2015, *BMJ* | 26063311 | 177 premarket approval studies for cardiovascular devices: 49% published. Of 152 FDA primary endpoints, 28% were unlabelled and 10% missing in publications. 11% of results were substantially different. |
|
||
| Hajibandeh S et al. 2015, *EJVES* | 26298221 | Vascular RCTs: share industry-sponsored rose from 6.67% to 28%. Methodological quality did not improve. |
|
||
| **Beard DJ et al. 2020, *Lancet* (ASPIRE)** | 32145797 | Guidance: surgical placebo controls are justified when evidence of efficacy is poor and there is a real risk of bias from placebo effects. Feasibility work is recommended. |
|
||
| **Al-Lamee R et al. 2018, *Lancet* (ORBITA)** | 29103656 | 200 patients randomized to PCI or placebo procedure. Exercise time difference 16.6 s (−8.9 to 42.0), not significant. Shows that a sham-controlled revascularization trial is feasible. |
|
||
| Natarajan P et al. 2024, *BMJ Open* | 38637129 | 61 placebo-control vs 38 non-operative-control surgical RCTs. Recruitment similar; attrition IRR 1.8 (1.1–3.0) for placebo trials. |
|
||
| Wall L et al. 2024, *Bone Joint J* | 39615522 | Discrete choice experiment. 31% of surgeons and 40% of patients would join no sham-controlled knee trial design. |
|
||
|
||
**Can TTE solve it? Yes for cost and for withholding treatment; partly for industry influence; no for sham and placebo questions.**
|
||
- **Cost:** TTE on existing claims or registry data costs a small fraction of an RCT. Griessbach 2024 shows that even trials save money by using routinely collected data.
|
||
- **Ethics of withholding treatment:** no one is assigned anything, so randomizing to "no procedure" is not an issue.
|
||
- **Industry influence:** independent investigators can run TTE on population data. However, claims do not identify devices well (file 01, limitation #10), so device-specific questions such as paclitaxel need registries or linked device identifiers.
|
||
- **Sham and placebo:** TTE cannot separate the specific effect of a procedure from the placebo effect. Hard outcomes in claims (amputation, death) are not prone to placebo effects. Symptom outcomes in EHR or registries are prone to them, and observational data cannot blind.
|
||
|
||
**Testable in a known-truth simulation?** Only partly. Cost and ethics are not testable. A placebo component could be added to a simulated PRO outcome that only a blinded trial removes, but it is outside the current DGP; this is optional.
|
||
|
||
---
|
||
|
||
## Summary: the most important new papers
|
||
|
||
1. **Shadbolt 2023 *JAMA Netw Open* (36648945) and Binko 2023 *Ann Vasc Surg* (37311508).** Only 14.6% of surgical RCTs finish on time and at full size. In vascular surgery, 22% are discontinued, mostly for poor enrollment, and 80% of those are never published.
|
||
2. **Wasmann 2019 *BMJ Open* (31619428), King 2005 *JAMA* (15741531) and PCRG 2008 *BMJ* (18977792).** Most eligible patients often refuse randomization, and those who refuse differ by covariates. Outcomes rarely differ, but preference may modify effects (ES about 0.16). These set the parameters for non-random consent in our simulated trial.
|
||
3. **Young 2004 *Surgery* (15523403) and Freedman 1987 *NEJM* (3600702).** Vascular surgeons' equipoise can be measured and is often absent, and preference strength predicts refusal to randomize. Clinical equipoise is the ethical standard.
|
||
4. **Javidan 2024 / Li 2023 *JVS* (37315910, 36572321) and Djerf 2020 *EJVES* (31866238).** Vascular RCT results hinge on 3–7 events, and procedure-related amputation within 1 year of revascularisation for claudication is 0.2%. Hard outcomes need population-scale data.
|
||
5. **Shwan 2024 *BJS Open* (39485886), Dhruva 2009 *JAMA* (20040556) and Hoel 2017 *JVS* (28222989).** Claudication trials use 541 different outcomes and few PROs, and 88% of device endpoints are surrogates. Linking VQI to Medicare restores long-term hard outcomes but not PROs.
|