Literature review 3: surgical RCT challenges vs TTE (merged from three verified drafts), draft Table 1, candidate simulation additions

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs/literature/drafts/B-surgeon-intervention.md

@@ -0,0 +1,271 @@
+# Draft B: surgeon and intervention challenges of surgical RCTs, and what TTE can do about them
+
+Scope: learning curve; skill and expertise variation; selection of surgeons and centres into trials; standardization and device heterogeneity; clustering by surgeon; blinding, sham and outcome assessment.
+
+All PMIDs were checked with NCBI E-utilities esummary on 2026-10-28. All numbers come from the PubMed abstracts fetched with efetch; full texts were not read. Papers already listed in `01-surgical-rct-limitations.md` are cited by PMID only and are not repeated in the tables.
+
+**Data sources for "TTE" in this draft.** TTE can run on three kinds of data, and they differ on exactly the problems in this chapter:
+- **Claims** (Medicare FFS): the operator NPI, CPT/HCPCS codes (vessel territory; angioplasty vs stent vs atherectomy), place of service, and objective outcomes (amputation, death, repeat procedures). There is no lesion anatomy, no device brand, no technical success, and no patient-reported outcomes.
+- **Clinical registries** (e.g., the Vascular Quality Initiative, VQI): de-identified physician and centre identifiers, lesion characteristics, devices, and technical details. Some outcomes are clinician-reported. Centres choose to join.
+- **Registry–claims linkage** (VQI–Medicare, "VISION"): registry procedure detail plus long-term claims follow-up. Mao 2022 (PMID 35181518) matched 93.0% of VQI patients to Medicare with 99.4% accuracy.
+
+---
+
+## 1. Learning curve
+
+**What it is.** Surgeons get better at a procedure the more often they do it, so outcomes depend on where on their learning curve the operator is. If a trial starts while a new procedure is still early on its curve, the new procedure looks worse than it will be in practice. If the trial waits until surgeons are proficient, equipoise may already be gone (Papachristofi 2016). Learning curves are measured and reported inconsistently.
+
+**How common / how big.**
+- A review of 592 learning-curve studies in minimally invasive abdominal surgery found that 86% used operative time as the proxy and only 6% used patient-oriented outcomes (Harrysson 2014).
+- An umbrella review of 62 reviews (1,990 studies) gave a definition of the learning curve in only 63% (so 37% gave none, by subtraction), 712 different data items were used, and 60 of the 62 reviews were of low or critically low quality (Ramirez 2026).
+- For radical prostatectomy, the learning curve did not plateau until about 250 prior operations. Predicted 5-year recurrence was 17.9% after 10 prior cases and 10.7% after 250 (Vickers 2007).
+- TAVR in the TVT registry: from the 1st to the 400th case, risk-adjusted mortality fell from 3.57% to 2.15% and bleeding from 9.56% to 5.08% (Carroll 2017).
+- Only 1 of 247 surgical RCTs analysed learning, and 2% used an expertise-based design (Conroy 2019; see Section 5).
+
+**Vascular example.**
+- **Carotid stenting in Medicare** (Nallamothu 2011): 1,792 new operators started after the CMS coverage decision. 30-day mortality was 2.3% for a new operator's 1st–11th procedures vs 1.4% from the 12th on (aOR 1.7, 1.2–2.4). This is a learning curve measured in claims through the operator identifier.
+- **Endovascular device trials** handle the learning curve with "roll-in" patients who are excluded from the randomized analysis. The IntraCoil femoropopliteal trial used 93 roll-in patients across 22 sites (Ansel 2002).
+
+| Paper | PMID | Takeaway (numbers from abstract) |
+|---|---|---|
+| Harrysson IJ et al. 2014, *Ann Surg* | 24670849 | 592 learning-curve studies. Operative time was the proxy in 86%, postoperative outcomes in 52%, patient-oriented outcomes in 6%. No increase over time in the complexity of analysis. |
+| Ramirez J et al. 2026, *Surg Innov* (umbrella review) | 42527319 | 62 reviews / 1,990 studies. A definition was given in 63%, surgeon expertise reported in 63%, 712 unique data items, and 60/62 reviews were low or critically low quality. |
+| Papachristofi O et al. 2016, *Trials* | 27245050 | Two-phase model that estimates initial skill, learning rate, final skill, and **duration of learning**. Frames the "when to run the definitive RCT" dilemma. |
+| Biau DJ et al. 2008, *Br J Surg* (LC-CUSUM) | 18498126 | Individual learning-curve monitoring. ERCP success rose from 82.0% (first 100) to 96.1% (last 129). Competence signalled at case 79. |
+| Vickers AJ et al. 2007, *JNCI* | 17652279 | 7,765 prostatectomies by 72 surgeons. Plateau at about 250 cases. 5-year recurrence 17.9% vs 10.7% (10 vs 250 prior cases). |
+| **Nallamothu BK et al. 2011, *JAMA*** | 21954477 | Medicare CAS, 24,701 procedures, 2,339 operators. Median operator volume was 3.0/year. Early vs late experience: 30-day mortality 2.3% vs 1.4% (aOR 1.7). Very-low vs high volume: aOR 1.9. |
+| Carroll JD et al. 2017, *JACC* (TVT Registry) | 28662805 | 42,988 TAVRs at 395 hospitals. Risk-adjusted mortality 3.57% → 2.15% from case 1 to 400. Vascular and bleeding risk higher in the first 100 cases. |
+| Ansel GM et al. 2002, *Catheter Cardiovasc Interv* | 12124949 | Femoropopliteal stent trial. 93 roll-in ("learning phase") patients across 22 sites, reported separately from the randomized comparison. |
+| Luft HS et al. 1987, *Health Serv Res* | 3112042 | Volume–outcome links reflect both "practice makes perfect" and "selective referral" (better surgeons attract more patients). The relative importance varies by procedure. |
+
+Already in 01: Cook 2004 (16279280), Ramsay 2000/2001 (11155830, 11319991).
+
+**Can TTE solve it? Partly.**
+- **Claims.** TTE does not remove the learning curve, but it changes which question is answered: it estimates the effect *as delivered* across the real mix of operator experience, not at a chosen point on the curve.
+- **Claims and registries.** Cumulative operator volume can be built from the operator identifier (NPI in claims, physician ID in VQI) and used as a time-varying covariate or effect modifier (Nallamothu 2011 shows this is feasible). Caveats:
+  - Medicare FFS volume is only part of an operator's total volume.
+  - Volume is left-truncated at the start of the data.
+  - Volume partly reflects selective referral (Luft 1987).
+- **Registries only.** These add technical success and procedural detail, which are closer to direct measures of proficiency.
+- **What TTE cannot do** is target "the effect once everyone is proficient" unless proficient operators can be identified and positivity holds among them.
+
+**Testable in a known-truth simulation? Yes.** Let operator skill rise with cumulative PVI volume (e.g., skill_j(t) = s_j + λ·log(1 + V_j(t))). This world shows three things that are specific to TTE:
+1. **Interference.** One patient's early PVI raises the operator's skill for later patients, which violates no-interference/SUTVA. The individual-level TTE contrast then differs from the effect of a population policy of "early PVI for all", which would also change volumes.
+2. **Strategy effects and skill are entangled.** In the delayed arm, PVIs happen later, when operators are more experienced. Part of any "benefit of waiting" is then skill gain, not timing. The simulation can split the two by comparing truths with λ = 0 and λ > 0.
+3. **Time-varying confounding by physician.** High-preference physicians build volume faster, so skill becomes correlated with preference *and* changes over time. A physician fixed effect cannot remove this; adjusting for cumulative volume might. The simulation can check whether a time-varying volume covariate recovers the truth.
+
+---
+
+## 2. Surgeon skill, expertise, and volume–outcome
+
+**What it is.** Surgeons differ in technical skill. In a conventional RCT, each surgeon does both procedures, often with unequal expertise ("differential expertise bias"; Devereaux 2005, in 01). Volume is a crude, widely used proxy for skill, and it does not separate skill from referral patterns.
+
+**How common / how big.**
+- **Video-rated skill.** In a meta-analysis of 7 studies (151 surgeons, 17,932 procedures), top- vs bottom-quartile skill was associated with less reoperation (OR 0.44) and fewer medical complications (OR 0.23) (Woods 2023). Peer-rated skill explained 25.8% of the variation in post-colectomy complications (Stulberg 2020).
+- **Volume.** A systematic review found that 71% of hospital-volume and 69% of physician-volume studies reported significant volume–outcome associations. AAA surgery was among the strongest, at a median of 3.3–13 excess deaths per 100 cases across the strongest conditions (Halm 2002).
+- **Reporting of expertise in trials.** Only 34.4% of surgical interventions in RCTs came with any data on operator expertise (Blencowe 2015, BJS; Section 4).
+
+**Vascular example.**
+- **PVI.** In the NCDR PVI registry (555 operators), high- vs low-volume operators had lower in-hospital MALE (OR 0.73) and MACE (OR 0.64), while hospital volume was not associated (Chunawala 2025).
+- **AAA.** Elective AAA repair: OR 0.66 for mortality at high-volume hospitals (Holt 2007).
+- **Carotid stenting.** In pooled trial data, operator *in-trial* annual volume mattered (≤3.2/year: aRR 2.30 vs >5.6/year) but lifetime experience did not (Calvet 2014).
+
+| Paper | PMID | Takeaway (numbers from abstract) |
+|---|---|---|
+| Stulberg JJ et al. 2020, *JAMA Surg* | 32838425 | 17 surgeons, video-rated skill 2.8–4.6 out of 5. Higher skill: complications 15.5% vs 20.6%. Skill explained 25.8% of the variation in complications. |
+| Woods MS et al. 2023, *JSLS* (meta-analysis) | 36818767 | 7 studies, 151 surgeons. Top vs bottom skill quartile: reoperation OR 0.44, any medical complication OR 0.23. Mortality not significant. Few studies exist. |
+| Halm EA et al. 2002, *Ann Intern Med* (systematic review) | 12230353 | 71% / 69% of hospital / physician volume studies were significant. Effect is larger for AAA and pancreatic surgery, and smaller for CEA and angioplasty. Clinical risk adjustment weakens associations. |
+| Holt PJ et al. 2007, *Br J Surg* (meta-analysis) | 17380547 | 421,299 elective AAA repairs. High-volume OR 0.66 (threshold 43/year). Ruptured: OR 0.78. |
+| Chunawala Z et al. 2025, *JAHA* (NCDR PVI) | 41147410 | 55,785 PVIs, 97 hospitals, 555 operators. Operator volume: MALE OR 0.73, MACE OR 0.64. Hospital volume: no significant association. |
+| Setogawa N et al. 2025, *J Endovasc Ther* (Japan) | 37675778 | Hospital EVT volume was not associated with 12-month amputation (Q4 vs Q1 OR 1.00), but was associated with death (OR 0.71). |
+| Calvet D et al. 2014, *Stroke* (CSTC pooled RCTs) | 24347422 | 1,546 CAS patients in EVA-3S/SPACE/ICSS. 30-day stroke or death 10.1% (low in-trial volume) vs 5.1% (high). aRR 2.30. Lifetime experience had no effect. |
+| Bednarska E et al. 2008, *Clin Orthop Relat Res* | 18446421 | Surgeon survey (n = 276): 53.4% willing to join an expertise-based RCT vs 18.3% a conventional RCT. |
+| Cook JA et al. 2018, *Trials* (qualitative) | 30189868 | Surgeons and methodologists: expertise-based designs help participation, but surgeons in each arm may differ systematically. Patient transfer between surgeons is a practical problem. |
+
+Already in 01: Devereaux 2005 (15637373), Cook 2015 (26025450), Birkmeyer 2013 (24106936), Birkmeyer 2003 (14645640).
+
+**Can TTE solve it? Partly.**
+- **What TTE gains.** It needs no "both-procedures" surgeon. Each operator does what they normally do, which resembles an expertise-based design: early-PVI operators and deferring operators are compared as they practise. This also means **skill is bundled with strategy and physician**.
+- **Claims.** These contain no skill measure. Volume and specialty are proxies, and the NPI enables random or fixed effects.
+- **Registries** add technical success and lesion complexity, which partly measure skill. Video-based skill is not available in any routine source.
+- **Remaining risk.** If skill is correlated with intervention preference, confounding by physician remains, unless within-physician comparisons are possible (see 02, B6).
+
+**Testable in a known-truth simulation? Yes.** Give physicians a latent skill correlated with preference. Compare three estimands: (i) the population as-practised effect (what TTE targets), (ii) the effect if every PVI were done by top-skill operators (an "expert trial"), and (iii) the effect under differential expertise.
+
+---
+
+## 3. Selection of surgeons and centres into trials (generalizability)
+
+**What it is.** Trials enrol centres and surgeons who are interested, have high volume, and are often credentialed with explicit audit thresholds. Their results can overstate the benefit, or understate the harm, of a procedure in routine practice. The classic case is carotid endarterectomy after NASCET/ACAS.
+
+**How common / how big.**
+- **Medicare CEA, 1992–93** (Wennberg 1998). Mortality was 1.4% at the 86 NASCET/ACAS trial hospitals vs 1.7%, 1.9%, and 2.5% at high-, average-, and low-volume non-trial hospitals. These were all above the trials' own 0.6% (NASCET) and 0.1% (ACAS).
+- **Hospitals that take part in trials** (acute coronary syndromes) had lower in-hospital mortality: 5.9% at non-participants vs 3.5% at high enrollers (Majumdar 2008).
+- **Counterpoint.** A Cochrane review found no consistent difference in outcomes between patients inside vs outside RCTs (63 of 73 comparisons not significant; Vist 2007). So the gap reflects *who delivers* care more than trial participation itself.
+- **Not all settings show a gap.** Population-based CEA in Olmsted County (Hallett 1998) and a statewide Kentucky registry (Mayo 1998; stroke plus death 2.8%) roughly matched trial results. However, Mayo's registry covered only 58% of the state's procedures and 10 of 17 hospitals.
+
+**Vascular example.**
+- **CEA:** Wennberg 1998 (above).
+- **Carotid stenting:** trial thresholds vs practice. Calvet 2014 recommends ≥6 CAS/year, while the median Medicare operator did 3.0/year (Nallamothu 2011).
+- **EVAR:** in Medicare (Schermerhorn 2008), the early advantage of EVAR (perioperative mortality 1.2% vs 4.8%) carried into practice, showing that population data can confirm the direction of trial results.
+
+| Paper | PMID | Takeaway (numbers from abstract) |
+|---|---|---|
+| **Wennberg DE et al. 1998, *JAMA*** | 9565008 | 113,300 Medicare CEAs. Trial hospitals 1.4% vs non-trial 1.7–2.5% mortality (trend p < .001). All were above the trials' 0.6%/0.1%. "Caution … translating efficacy … to effectiveness". |
+| Hallett JW et al. 1998, *J Vasc Surg* | 9620136 | Olmsted County, all CEAs 1970–95 (n = 297 patients). Freedom from ipsilateral stroke at 2 years was 97% vs NASCET 91%. Higher 30-day stroke rate in patients over 80. |
+| Mayo SW et al. 1998, *J Vasc Surg* | 9652463 | Kentucky statewide registry: stroke plus death 2.8%, comparable to NASCET/ACAS. Covered 58% of CEAs. Surgeon stroke rates ranged 0–10%. |
+| Vist GE et al. 2007, *Cochrane* | 17443630 | 55 studies, 31,140 trial vs 20,380 non-trial patients. 63 of 73 comparisons not significant. Participation itself is not associated with better or worse outcomes. |
+| Majumdar SR et al. 2008, *Arch Intern Med* | 18362259 | Non-ST-elevation acute coronary syndromes (non-surgical). In-hospital mortality 5.9% / 4.4% / 3.5% by trial-enrolment tertile. High-enrolling hospitals aOR 0.8. |
+| Schermerhorn ML et al. 2008, *NEJM* | 18234751 | 22,830 matched pairs in Medicare. EVAR vs open perioperative mortality 1.2% vs 4.8%. By 4 years: rupture 1.8% vs 0.5% and AAA reintervention 9.0% vs 1.7%. Practice data add the late harms that short trials miss. |
+| Mao J et al. 2022, *J Vasc Surg* | 35181518 | VQI–Medicare indirect linkage: 93.0% matched, 99.4% accuracy. Registry procedure detail plus claims follow-up. |
+
+Already in 01: Rothwell 2005/2006 (15639683, 16871331), Cutteridge 2025 (40105904), Jones 2026 (42526742).
+
+**Can TTE solve it? Yes for claims, partly for registries.**
+- **Claims.** Medicare FFS includes essentially every operator who bills Medicare, so no surgeons or centres are selected in. The TTE estimand is the effect as practised, which is what policy needs.
+- **VQI.** Centres choose to join, and they tend to be engaged in quality improvement. Registry TTEs therefore carry a milder version of the trial's centre selection. VISION linkage helps with follow-up but not with which centres take part.
+- **What TTE cannot give** is the "ideal-operator" effect, unless operators are restricted to high-volume ones (a transportability or subgroup analysis).
+
+**Testable in a known-truth simulation? Yes.** Run a simulated RCT restricted to the top skill/volume quintile of physicians and compare its estimate with the population truth and with TTE on everyone. The gap is the generalizability bias, and TTE should close it.
+
+---
+
+## 4. Standardization, quality control, technique variation, device heterogeneity
+
+**What it is.** A surgical "treatment" is a bundle of steps, devices, and peri-operative care. Trials must decide how much to standardize. Too little, and nobody knows what was tested; too much, and the result may not transfer to practice. In endovascular trials, "PVI" can mean plain balloon, stent, drug-coated device, or atherectomy, in different vessel territories, and these may not share one effect.
+
+**How common / how big.**
+- Of 160 interventions in surgical RCTs, standardization was mentioned for only 29.4%, and only 28% of articles measured adherence (Blencowe 2015).
+- In placebo-controlled surgical trials, only 7% attempted to standardize and 4% monitored delivery (Cousins 2020; Section 6).
+- In arteriovenous-graft RCTs, quality assurance had "not improved over the past three decades", and no trial assessed technical performance (Kingsmore 2025).
+
+**Vascular / claudication example.**
+- **Tibial PVI for claudication.** In Medicare 2017–19, 27.7% of index claudication PVIs included tibial treatment. The physician-level rate had a median of 20.0% (IQR 9.1–37.5%) and was associated with early-career status (aOR 2.97) and office-based practice (aOR 2.37) (Bose 2023). Infrapopliteal vs femoropopliteal PVI had a higher risk of CLTI (aHR 1.31) and amputation (aHR 1.72) (Bose 2025). The "intervention" is therefore technically heterogeneous, and the heterogeneity varies by physician.
+- **Devices.** A network meta-analysis of femoropopliteal devices found that technical success and restenosis differed substantially by device class (Katsanos 2014).
+
+| Paper | PMID | Takeaway (numbers from abstract) |
+|---|---|---|
+| **Blencowe NS et al. 2015, *Br J Surg*** | 26041565 | 80 surgical RCTs / 160 interventions. Described 80.6%, standardization mentioned 29.4%, adherence measured in 28% of articles, operator expertise reported for 34.4%. Pragmatic and explanatory trials were similar. |
+| Blencowe NS et al. 2016, *Br J Surg* | 27462835 | Typology for standardizing surgical interventions: purpose → components and steps → monitoring of deviations. |
+| Butterworth JW et al. 2021, *Eur J Surg Oncol* | 33059943 | 34 oncology studies. Quality-assurance challenges fall into credentialing (volume is a weak criterion), standardization (variation between centres), and monitoring. |
+| Kingsmore DB et al. 2025, *J Vasc Access* | 38501338 | Vascular-access (AV graft) RCTs: quality assurance highly variable, no pre-trial monitoring, none assessed technical performance. |
+| Katsanos K et al. 2014, *J Vasc Surg* (network meta-analysis) | 24661896 | 16 femoropopliteal RCTs. Technical success OR 13.6 for covered stents vs balloon. Target lesion revascularization RR 0.36 for paclitaxel-coated balloons. Device classes are not interchangeable. |
+| **Bose S et al. 2023, *J Vasc Surg*** | 36058433 | Medicare claudication PVI (n = 59,930; 1,542 physicians). Tibial PVI 27.7%. Physician-level rate IQR 9.1–37.5%. Early career aOR 2.97, office-based aOR 2.37. |
+| Bose S et al. 2025, *Eur J Vasc Endovasc Surg* | 38906366 | Medicare claudication PVI (n = 36,147). Infrapopliteal vs femoropopliteal: CLTI aHR 1.31, repeat PVI aHR 1.12, amputation aHR 1.72. |
+| Sansosti AA et al. 2024, *J Vasc Surg* (VQI) | 38944400 | Femoropopliteal atherectomy. Embolic protection used in 40% (hospital) vs 4.4% (office). Office setting had more reintervention at 2 years. Technique varies by setting. |
+
+Already in 01: Boutron 2017 (28630973), Garas 2018 (29110271), Jones 2026 (42526742).
+
+**Can TTE solve it? Partly.**
+- **TTE cannot standardize anything.** The "early PVI" strategy is whatever PVI was actually done.
+- **Claims.** CPT codes record vessel territory and modality (angioplasty, stent, atherectomy) and the setting, so the strategy can be defined more finely, or modality treated as a version of treatment. Device brand, drug coating (only partly, via HCPCS pass-through codes), lesion length, and technical success are not recorded.
+- **Registries** (VQI) record lesion characteristics, devices, and technical details, so technique variation is largely measurable there, and "versions of treatment" can be modelled explicitly.
+- **The open question** is whether a single "early PVI" effect is meaningful when the technique mix differs by physician. This is a consistency (well-defined intervention) problem.
+
+**Testable in a known-truth simulation? Yes.** Give each physician a probability of a high-risk technique (e.g., tibial PVI) with its own effect. Then compare a TTE that ignores versions (estimating the effect of the as-practised mixture) with one stratified on modality.
+
+---
+
+## 5. Clustering of outcomes by surgeon
+
+**What it is.** Patients treated by the same surgeon or centre have correlated outcomes, even under individual randomization. Ignoring this understates the standard errors and, when provider effects are tied to volume or treatment, can bias the effect estimate. Few trials handle it.
+
+**How common / how big.**
+- **Trials rarely adjust.** Of 68 large interventional RCTs, 16% reported the number of providers, 9% stratified randomization by provider, and 4% adjusted the analysis for provider (Biau 2008, *JCE*).
+- **The picture is little better ten years later.** Of 247 surgical RCTs, 40% of trials stratified by surgeon did not adjust the analysis for surgeon (Conroy 2019).
+- **Re-analysis changes results.** Re-analysing three surgical RCTs found significant provider effects. In the hip-replacement trial, the treatment effect *became* significant after adjusting for provider and volume (Biau 2008, *Ann Surg*).
+- **Size of clustering.** Cook 2012 reported ICCs for 198 outcomes from 10 trials and concluded that clustering is "more of an issue than has been previously acknowledged" (no single pooled ICC in the abstract).
+
+**Vascular example.** Physician-level variation in claudication PVI technique is wide (tibial PVI rate IQR 9.1–37.5%; Bose 2023), and the analysis used hierarchical models. In NCDR, 55,785 PVIs were performed by 555 operators, an average of about 100 per operator (Chunawala 2025). Clustering in claims PVI data is therefore substantial in both technique and volume.
+
+| Paper | PMID | Takeaway (numbers from abstract) |
+|---|---|---|
+| **Biau DJ et al. 2008, *J Clin Epidemiol*** | 18394535 | 68 multicentre interventional RCTs (n ≥ 200). Number of providers reported 16%. Stratified by provider 9%. Analysis adjusted for provider 4% and for centre 6%. |
+| **Biau DJ et al. 2008, *Ann Surg*** | 18438129 | Patient-level re-analysis of 3 RCTs (22–88 providers each). Significant provider effects (ophthalmology p < .0001; orthopaedics p = .0037). Treatment × volume interaction. Adjustment changed the conclusion. |
+| Cook JA et al. 2012, *Trials* | 22217216 | ICCs for 198 outcomes in 10 surgical trials. Median cluster size 32 (centre) and 6 (surgeon). ICCs varied widely by outcome type. |
+| **Conroy EJ et al. 2019, *J Clin Epidemiol*** | 30439544 | 247 surgical RCTs. Expertise-based design 2%, credentials predefined 39%, only 1 trial analysed learning. One-third of centre-stratified and 40% of surgeon-stratified trials did not adjust the analysis. |
+| Conroy EJ et al. 2019, *J Clin Epidemiol* (funded applications) | 31121302 | 50 funded UK trials. 60% of surgeon-stratified trials planned to adjust. One-third of referee responses involved funder-driven changes on learning or clustering. |
+| Conroy EJ et al. 2022, *Trials* (guidance review) | 36221107 | 28 guidance documents. Clustering by centre is covered well, by surgeon poorly. Analysis methods for clustering in 5 documents and for learning in 4. No single guidance document exists. |
+
+Already in 01: Lee & Thompson 2005 (15649931, 16279138), Roberts 2005 (16279137), Walters 2010 (20492055). Analysis methods in 02 B6 (Panageas, Kahan & Morris, within-physician designs).
+
+**Can TTE solve it? Yes for inference, partly for confounding.**
+- **Claims and registries** both have operator identifiers (NPI; VQI physician ID), so cluster-robust SEs, random effects, physician fixed effects, or within-physician designs are standard.
+- **Clustering is larger in TTE than in a trial**, because physicians choose the strategy. Physician-level clustering therefore becomes **confounding by physician** (preference correlated with outcome style), not just a precision problem (see 02 B6).
+- **Within-physician comparisons** need physicians who sometimes intervene early and sometimes defer.
+
+**Testable in a known-truth simulation? Yes.** Random intercepts and random slopes by physician, with the intercept correlated with intervention preference. Check (a) the coverage of naive vs cluster-robust vs mixed-model SEs and (b) bias with and without within-physician adjustment.
+
+---
+
+## 6. Blinding, sham control, and outcome-assessment (detection) bias
+
+**What it is.**
+- Surgeons cannot be blinded, patients rarely are, and outcome assessors often are not.
+- Unblinded subjective outcomes (pain, symptom scores, self-limited walking distance) are prone to expectation and observer bias.
+- Sham-controlled trials of invasive cardiovascular procedures have repeatedly shrunk effects seen in open-label trials.
+- Observational data have their own version of the problem. Outcomes are recorded when patients are looked for (surveillance bias), and diagnoses are coded by the treating physician.
+
+**How common / how big.**
+- **Non-blinded assessors exaggerate effects.** By 36% for subjective binary outcomes (Hróbjartsson 2012) and by 68% for measurement-scale outcomes (Hróbjartsson 2013). In an updated 2025 analysis the figure was 29% overall and larger in non-drug trials (ratio of odds ratios 0.62; Salazar 2025).
+- **The bias is in subjective outcomes, not objective ones.** In a meta-epidemiological study, lack of blinding biased subjective outcomes (ROR 0.75) but not objective ones (ROR 1.01) (Wood 2008).
+- **Blinding is under-used in surgery.** In 378 surgical RCTs only 24.0% of patients were blinded. Failure to blind surgeons where it was feasible was associated with more significant results (OR 13.7; very wide CI 1.3–142.8) (Probst 2019).
+
+**Vascular / claudication example.**
+- **Sham analogue for claudication.** The closest is ORBITA, which tested revascularization for an exertional symptom against a sham procedure. PCI did not improve exercise time beyond placebo (16.6 s, −8.9 to 42.0). ORBITA-2, in patients off antianginals, did show a symptom benefit (OR 2.21).
+- **Renal denervation.** SYMPLICITY HTN-3 found that the large blood-pressure falls in unblinded studies largely disappeared against sham (difference −2.39 mmHg, not significant).
+- **Claudication trials.** Claudication revascularization trials (ERASE, CLEVER, IRONIC, SWEDEPAD 2; in 01) use unblinded walking distance or quality of life.
+- **Claims outcomes.** The usual claims endpoints (amputation, death, repeat PVI) are objective in Wood's sense. Even so:
+  - ICD-10 coding distinguishes claudication from CLTI with only 81.3% agreement against VQI (Bose 2025).
+  - About half of PVI patients get no surveillance imaging in year 1 (Brooke 2018), so restenosis and repeat PVI depend on who looks.
+
+| Paper | PMID | Takeaway (numbers from abstract) |
+|---|---|---|
+| **Hróbjartsson A et al. 2012, *BMJ*** | 22371859 | 21 trials with both blinded and non-blinded assessors. Non-blinded assessors exaggerated the OR by 36% (ROR 0.64), driven by misclassifying a median of only 3% of patients. |
+| Hróbjartsson A et al. 2013, *CMAJ* | 23359047 | Measurement-scale outcomes, 16 trials. Non-blinded assessors exaggerated effect size by 68% (95% CI 14–230%). |
+| Salazar J et al. 2025, *J Clin Epidemiol* (update) | 40258524 | 66 trials. Pooled ROR 0.71, about a 29% exaggeration. Larger in non-drug trials (ROR 0.62) and industry-funded trials (0.57). |
+| **Wood L et al. 2008, *BMJ*** | 18316340 | Lack of blinding: ROR 0.75 for subjective outcomes vs 1.01 for objective outcomes. Little bias for all-cause mortality. |
+| Probst P et al. 2019, *Langenbecks Arch Surg* | 30824993 | 378 surgical RCTs. 24.0% of patients blinded. Assessor blinding was possible in 69%. Not blinding feasible surgeons was associated with significant results (OR 13.7). |
+| Cousins S et al. 2020, *J Clin Epidemiol* | 31786153 | 96 placebo-controlled surgical RCTs. 68% randomized fewer than 100. 34% gave no rationale for the placebo. Standardization 7%, monitoring 4%. |
+| **Al-Lamee R et al. 2018, *Lancet* (ORBITA)** | 29103656 | PCI vs sham in stable angina, n = 200. Exercise-time difference 16.6 s (−8.9 to 42.0), not significant. |
+| Rajkumar CA et al. 2023, *NEJM* (ORBITA-2) | 38015442 | n = 301, off antianginals. Angina score 2.9 vs 5.6 (OR 2.21). A sham can show *real* benefit. |
+| Bhatt DL et al. 2014, *NEJM* (SYMPLICITY HTN-3) | 24678939 | n = 535. Systolic BP −14.1 vs −11.7 mmHg with sham. Difference −2.39 (−6.89 to 2.12). |
+| **Bose S et al. 2025, *Circ Cardiovasc Qual Outcomes*** | 40439664 | VQI-VISION, n = 54,180 PVIs. ICD-10 claudication vs CLTI: sensitivity 80.9%, specificity 81.9%, agreement 81.3%. Machine learning raised the AUC to 0.892. |
+| Bilimoria KY et al. 2013, *JAMA* | 24100354 | Surveillance bias: hospital VTE rates rose from 5.0 to 13.5 per 1,000 across quartiles of imaging use. More looking, more events found. |
+| Brooke BS et al. 2018, *JAMA Surg* | 29049809 | Medicare. Only 52–58% of patients had imaging follow-up within 1 year after bypass or PVI. Joining VQI did not change this (RR about 0.99). |
+| Jacob-Brassard J et al. 2021, *Clin Invest Med* (Ontario) | 34152705 | Administrative PAD revascularization codes: PPV 94.3%, or 98.1% with physician claims. Procedures are coded accurately. |
+
+Already in 01: Wartolowska 2014/2016 (24850821, 27008687), Boutron 2017 (28630973).
+
+**Can TTE solve it? No for blinding, partly for outcome bias.**
+- **No blinding.** TTE cannot blind anyone.
+- **Claims.** These have no patient-reported outcomes or walking distance, so the subjective endpoints most open to bias are simply not available. The objective claims endpoints (death, major amputation) are the ones Wood 2008 found free of blinding bias. Physician-recorded endpoints remain at risk:
+  - CLTI diagnosis codes (81% agreement with the registry).
+  - Repeat PVI, which is itself a physician decision.
+  - Restenosis, which depends on surveillance (Brooke 2018).
+- **Registries** record walking or symptom status and patency, but these are clinician-reported and unblinded. Registries add detail without solving detection bias.
+- **Practical rule for the paper:** make death and major amputation primary; treat CLTI and repeat PVI as at risk of detection or decision bias.
+
+**Testable in a known-truth simulation? Yes, optionally.** Make the probability that CLTI is *recorded* (or that repeat PVI is triggered) depend on prior PVI or physician style (differential ascertainment), and keep amputation and death recorded perfectly. TTE should be unbiased for the objective outcomes and biased for the ascertainment-dependent ones. Blinding itself has no counterpart in the simulation.
+
+---
+
+## Table 1 inputs (one line each)
+
+| Challenge | Evidence it is common | Vascular example | TTE: claims | TTE: registry (VQI/VISION) | Simulation |
+|---|---|---|---|---|---|
+| Learning curve | Plateau at about 250 cases (Vickers); 1/247 RCTs analysed learning (Conroy) | CAS Medicare early-experience aOR 1.7 (Nallamothu) | Partly: cumulative NPI volume | Partly: plus technical success | Yes. Interference, delayed arm treated by more skilled operators, time-varying physician confounding |
+| Skill and expertise | Skill explains 25.8% of complication variation (Stulberg) | PVI operator volume MALE OR 0.73 (Chunawala) | Partly: proxies only | Partly | Yes: latent skill × preference |
+| Surgeon and centre selection | CEA trial vs non-trial hospitals 1.4% vs up to 2.5% (Wennberg) | CEA, CAS volume thresholds | Yes: all operators | Partly: centres self-select | Yes: "expert-only RCT" vs population |
+| Standardization and devices | 29.4% of trial interventions standardized (Blencowe) | Tibial PVI physician IQR 9–38% (Bose 2023) | Partly: CPT modality and territory | Mostly: lesion and device data | Yes: versions of treatment |
+| Clustering | 4% of RCTs adjusted for provider (Biau) | 555 operators in NCDR PVI | Yes (inference); confounding remains | Yes | Yes: random intercepts and slopes |
+| Blinding and detection | Non-blinded assessors +36% / +68% (Hróbjartsson) | ORBITA-type sham; CLTI code agreement 81% | No blinding; objective endpoints are fine | No; clinician-reported | Optional: differential ascertainment |
+
+---
+
+## Summary: most important new papers
+1. **Wennberg 1998 *JAMA* (9565008):** the canonical trial vs non-trial centre gap for CEA in Medicare. It anchors the argument that claims-based TTE estimates the effect as practised.
+2. **Nallamothu 2011 *JAMA* (21954477):** a learning curve and volume effect measured *in Medicare claims* through operator identifiers, for an endovascular procedure. It shows that operator experience can be modelled in claims-based TTE.
+3. **Biau 2008 *JCE* / *Ann Surg* (18394535, 18438129) and Conroy 2019 *JCE* (30439544):** surgical RCTs almost never account for provider effects or learning, and doing so can change conclusions.
+4. **Hróbjartsson 2012 *BMJ* (22371859) and Wood 2008 *BMJ* (18316340):** unblinded subjective outcomes are inflated by about 30–70%, while objective outcomes are not. This justifies TTE using death and amputation and is cautious about walking distance and CLTI coding.
+5. **Bose 2023 *JVS* (36058433) and Bose 2025 *CCQO* (40439664):** in claudication PVI, technique varies widely by physician, and the claims coding of CLTI is about 81% accurate against VQI. These define the "versions of treatment" and outcome-misclassification limits for claims vs registry TTE.