Per the Tool's rule, any single High warrants a detailed hazard/harm/probability assessment before deployment. The Highs cluster around two honest themes: clinical proximity + consequence (the system sits in the care-planning path for a high-stakes disease), and the not-yet-built operational layer (runtime monitoring / audit trail). Neither is a reason not to proceed; both define the mitigation and build agenda.
High: LPS-1 Distance From Patient · LPS-3 Consequences of Failure · LPS-5 Monitoring Difficulty · LPS-9 Population/Disparity · T&D-9 Monitoring & Incident Detection. Deliberate Lows (design strengths): LPS-2 Decision Autonomy · T&D-1 Sensitive Data (synthetic) · T&D-10 Detection & Traceability.
| # | Risk modifier | Response | Rationale / evidence | CRRF mitigation (Tier 2) |
|---|---|---|---|---|
| 1 | Distance From Patient | High | The framework sits inside clinical reasoning — trajectory placement, gap detection, next-best-action for nephrology care. This is "semi-direct involvement in patient care… used by a healthcare professional as part of broader clinical judgment," the CHAI High definition. | Gate 7 — authority is earned per output, never presumed by proximity |
| 2 | Decision Autonomy | Low | Assistive by explicit design: "It should not behave like an autonomous prescriber" (§24). AI informs/recommends; the human reviews and decides; risk rests on the human reviewer. A deliberate, documented choice. | Gate 7 + six-tier ladder caps outputs at advisory (T4) absent clean passage |
| 3 | Consequences of Failure | High | Worst-case severity in CKD/ESKD is high: a missed sustained stage-4 designation, an unsafe renal dose carried forward, or delayed access planning can contribute to permanent harm. Human review lowers probability, not worst-case severity. | Gates 4–5 — interpret before use; abstain if insufficient |
| 4 | Use Context & Complexity | Medium | Use context is outpatient nephrology follow-up (non-emergency) — argues Low — but the population is medically complex and often unstable (multi-comorbid CKD, MGUS, HF, progressing stage 4+). The complexity axis pulls it to Medium. | Gate 1 — declare the question and the patient's context up front |
| 5 | Level of Difficulty Monitoring Output | High | No real-time monitoring exists; longitudinal narrative output is hard to monitor at scale. Framework §21 lists "human override and audit trail" as a known gap. Per CHAI, monitoring "needs to be developed before implementation" = High. | Gate 9 — learn under governance (principle present; tooling is the build item) |
| 6 | React Time | Medium | Most outputs feed longitudinal planning reviewed at 3–4-month cadence, giving reaction time (argues Low). But some surfaced signals (e.g., critical hyperkalemia, acute deterioration) would need faster response — Medium is the honest blend. | Gate 8 — outputs ship with reversal conditions the clinician can act on |
| 7 | Breadth of Potential Harm | Medium | Affects a moderate CKD population at pilot scope. Note: breadth scales with deployment — enterprise-wide rollout would move this toward High and should be re-scored at that decision point. | Gate 6 — cross-stratum coherence before any population-level claim |
| 8 | Cross-System Propagation Risk | Medium | The External Data Reconciliation Agent ingests ADT, discharge summaries, outside labs, and claims, so the system is integrated — but outputs are advisory and clinician-gated, which limits automated cascade. Integrated, limited connections = Medium. | Gate 3 — provenance/currency checks on every ingested feed |
| 9 | Population Sensitivity / Disparity Risk | High | CKD carries documented equity hazards: transplant-referral disparities, historical eGFR race adjustment, and unequal access to modality planning. A tool touching candidacy and access could reinforce inequities if not equity-gated. Significant disparity potential = High. | Gate 6 — subgroups must cohere; Gate 8 exposes disparity-relevant assumptions |
Several Technology & Data modifiers are architecture-dependent. Because the framework is a pre-production demonstration on synthetic data, the honest current rating is Low, with an explicit Need More Info flag marking where the score must be re-taken against the production architecture — using CHAI's own abstention affordance rather than asserting a Low that the production system has not earned.
| # | Risk modifier | Response | Rationale / evidence | CRRF mitigation (Tier 2) |
|---|---|---|---|---|
| 1 | Use of Sensitive Data | Low→ High at production | Runs entirely on a synthetic cohort with documented ground truth — CHAI's Low definition ("synthetic, nonconfidential"). On real PII/PHI this becomes High. The synthetic-first design is a deliberate risk-lowering choice worth surfacing to governance. | Gate 3 — sensitive-data handling is part of "what we have" |
| 2 | Accuracy of Data | Medium | Designed to consume messy real-world coding (it explicitly harmonizes heterogeneous proteinuria measures and labels med-list confidence). Synthetic data is controlled today, but the operational target is Medium-accuracy source data. | Gate 4 — a code is not a phenotype until validated |
| 3 | Completeness of Data | Medium | Longitudinal CKD data is notoriously fragmented across systems. The framework's premise is to detect and flag gaps rather than paper over them — a mitigant that keeps this at Medium rather than High. | Gate 3 — completeness/gaps vs. what the question expects |
| 4 | Veracity of Data | Medium | External feeds (claims, outside labs, ADT) have variable provenance and refresh cadence. The data-confidence model (Confirmed/Probable/Conflicting/Missing/Outdated) is the internal control. | Gate 3 — provenance, currency, trustworthiness |
| 5 | Data Transparency | Low→ Med at production | The synthetic cohort is fully documented (canonical Patients repository + ground truth), so provenance is currently complete. Production data-source documentation would need to be assembled — re-score then. | Gate 8 — provenance is shown, not assumed |
| 6 | Sufficiency & Representativeness | Medium | 18 synthetic patients are deliberately diverse but small and not a population sample; adequate for demonstration, not for generalization or model training. Representativeness of the operational data is the real test. | Gate 2 + Gate 6 — declare needed data; check cohort coherence |
| 7 | AI Model Security Vulnerabilities | Low→ Med at production | Today a self-contained, application-bound HTML demo with no external interfaces (CHAI Low). An EHR-integrated, internal-facing production deployment would be Medium — a decision-point re-score. | Gate 9 — governed deployment surface |
| 8 | AI Model Lifecycle Management | Medium | Strong version control (git repo, changelog, versioned specs) — but no formal retraining/validation/rollback governance for a deployed model. The system is largely rule/heuristic-based, lowering drift risk but not removing the need for governed updates. | Gate 9 — versioned, auditable knowledge layer |
| 9 | AI Monitoring, Incident Detection & Response | High | No runtime monitoring, anomaly detection, or clinician-override audit trail is built (§21 gap). CHAI High: "monitoring is minimal or absent… no clear separation between operational incidents and AI safety incidents." This is the single most actionable pre-deployment gap. | Gate 9 — the gate whose tooling must be built to clear this |
| 10 | AI Detection & Traceability | Low | Traceability is a design default: rationale visibility on hover/click, explicit data-confidence labels, and separation of observation / interpretation / action. AI influence is legible to the clinician end-to-end — CHAI Low. | Gate 8 — "show the work" is the framework's core output contract |