What the QSR + GMLP Documentation Gap Audit actually produces

(1) SUBPART B (Quality System, 21 CFR 820.20) — General QMS structure, management responsibility, design/risk management integration. Current state: ISO 13485-aligned QMS in place per SOP library; QMSR alignment in progress. Gap: QMSR sections on AI/ML-specific governance (algorithm change control, retraining protocol) not finalized. P1 finding. (2) SUBPART C (Design Controls, 21 CFR 820.30) — Design inputs/outputs, design review, design verification/validation, design transfer. Current state: 510(k)-cleared design history file (DHF) exists; IEC 62304 software lifecycle documented. Gap: Algorithm change protocol drafted but not approved; post-clearance AI-specific design changes (model retraining, threshold adjustment) not formally controlled. P0 finding (blocks algorithm updates). (3) SUBPART E (Purchasing Controls, 21 CFR 820.50) — Supplier/vendor qualification, including cloud infra and third-party annotation services. Current state: Unclear from input. Gap: Third-party labeling/annotation data provenance not documented; cloud vendor cybersecurity attestation (SOC2, ISO27001) required. P1 finding. (4) SUBPART F (Identification and Traceability, 21 CFR 820.60) — Model versioning, training-data lineage, software build traceability. Current state: Model cards maintained per MLOps policy. Gap: Clinical validation study (n=2,400) report exists but lacks disaggregated performance (subgroup analysis by ethnicity, age, gender per recent FDA AI guidance); training-data lineage incomplete. P0 finding. (5) SUBPART I (Acceptance Activities, 21 CFR 820.86) — Pre-release testing, acceptance criteria, test protocols. Current state: Clinical study (n=2,400) report on file. Gap: Algorithm change and retraining acceptance criteria not formally specified; post-market surveillance of model drift not active. P1 finding. (6) SUBPART M (Records, 21 CFR 820.180) — Complaint handling, trend analysis, MDR reporting. Current state: Complaint system in place. Gap: Post-market complaint/adverse-event data not linked to algorithmic performance metrics (e.g., does a spike in false-negative diabetic retinopathy detections correlate with specific patient subgroups?). P1 finding.

(1) MULTIDISCIPLINARY EXPERTISE — Evidence: Clinical study team (ophthalmologists, ML engineers). Gaps: Software security specialist not named; clinical informatics support for EHR integration not mentioned. Partial alignment. (2) GOOD SOFTWARE ENGINEERING & SECURITY PRACTICES — Evidence: ISO 13485 lifecycle in place. Gaps: FDA premarket cybersecurity guidance alignment not documented; no mention of penetration testing, API security, or cloud-infrastructure hardening. P1 gap. (3) REPRESENTATIVE CLINICAL STUDY PARTICIPANTS — Evidence: Study n=2,400. Gap: Demographic breakdown (age, ethnicity, gender, comorbidities) not provided in input; geographic diversity (single center vs multi-center) unclear. Representativeness cannot be confirmed. P0 gap if study is not disaggregated. (4) INDEPENDENT TRAINING & VALIDATION DATASETS — Evidence: Assumed independent (standard practice). Gap: Training-data source (proprietary vs public fundus-image database?) and contamination risk not documented. P1 gap. (5) REFERENCE STANDARD BASED ON BEST AVAILABLE METHODS — Evidence: Input states predicate K213054 (IDx-DR) used. Gap: Whether current study matches predicate's clinical reference standard (clinical grading by certified ophthalmologist vs automated predicate comparison) not specified. P1 gap. (6) MODEL DESIGN TAILORED TO DATA & INTENDED USE — Evidence: Retinal-image-specific architecture (assumed deep learning on fundus photos). Gaps: Model architecture not detailed; adversarial robustness (e.g., poor performance on non-fundus-camera images, low-contrast images) not tested. P1 gap. (7) FOCUS ON PERFORMANCE OF HUMAN-AI TEAM — Evidence: Intended use states 'assists clinicians, does not replace judgment.' Gap: Human-factors study (does clinician use the output correctly? Does it reduce or increase diagnostic error?) not mentioned. P1 gap per IEC 62366 usability requirements. (8) TESTING DURING CLINICALLY RELEVANT CONDITIONS — Evidence: n=2,400 study completed. Gap: Subgroup performance (pediatric patients, non-dilated fundus exams, comorbid conditions) not disaggregated. P0 gap if pediatric data unavailable. (9) USERS PROVIDED WITH CLEAR ESSENTIAL INFORMATION — Evidence: Labeling exists but unchanged since 2022 clearance. Gap: Algorithm change (if any) not reflected in labeling; confidence intervals or uncertainty bounds not provided; no mention of failure modes (image quality too poor, atypical presentation). P1 gap. (10) DEPLOYED MODELS MONITORED — Evidence: Post-market surveillance plan on file but lacks AI-specific metrics. Gap: No active monitoring of model drift (performance degradation over time as real-world data distribution shifts away from training data); no mechanism to detect demographic-specific performance drops (e.g., worse performance in pediatric patients if retraining occurs). P0 gap.

P0 FINDINGS (Pre-deployment blockers): (1) Algorithm Change Protocol Not Approved — Regulatory citation: 21 CFR 820.30(j) (design changes must follow design-change procedures; IEC 62304 requires change documentation). Expected documentation: Formal algorithm change control SOP including risk assessment, verification, validation, and clinical re-evaluation triggers. Current state: Protocol drafted, not finalized/approved. Recommended next step: Engage regulatory affairs + quality to finalize and approve SOP; submit to external regulatory counsel for alignment with FDA expectations on post-market algorithm updates. Owner: Regulatory Affairs. P1 FINDINGS (Required for QSR/QMSR completeness): (1) Subgroup Performance Data Missing — Regulatory citation: 21 CFR 820.30(g) (design validation must demonstrate safety/effectiveness); FDA AI/ML guidance (2019-2023) emphasizes disaggregated performance by demographic and clinical strata. Expected documentation: Validation study report with sensitivity/specificity by age group, ethnicity, disease stage, image quality. Current state: Study report (n=2,400) exists but disaggregation not confirmed in input. Recommended next step: Request full study report from clinical team; if disaggregated data exists but not documented, add to technical file; if missing, conduct subset analysis or post-market study plan. Owner: Clinical + Regulatory Affairs. (2) Cybersecurity per FDA Premarket Guidance — Regulatory citation: 21 CFR 820.30(h) (risk management); FDA Premarket Cybersecurity Guidance (January 2023). Expected documentation: Threat modeling, penetration-testing results, cloud-infrastructure SOC2/ISO27001 attestation, API security architecture. Current state: ISO13485 covers some elements; cloud-vendor attestation not mentioned. Recommended next step: Conduct cybersecurity risk assessment per FDA framework; obtain SOC2 Type II or ISO27001 audit from cloud vendor; document in technical file. Owner: Engineering + Regulatory Affairs. (3) Post-Market Surveillance Plan AI-Specific Metrics — Regulatory citation: 21 CFR 820.100 (post-market surveillance), Article 72 EU AI Act (post-market monitoring); FDA AI/ML guidance on performance monitoring. Expected documentation: Plan specifying real-world performance metrics (sensitivity/specificity by patient cohort, image quality, false-negative rate trends, complaint correlation to algorithm version). Current state: General post-market surveillance plan on file; AI-specific metrics not outlined. Recommended next step: Update surveillance plan to include drift detection (quarterly performance check), subgroup analysis (age/ethnicity/disease-stage stratification), and escalation triggers (e.g., if sensitivity drops >5% in any cohort). Owner: Quality + Regulatory Affairs. (4) Training Data Lineage & Labeling Documentation — Regulatory citation: 21 CFR 820.25(b) (documentation and record retention); FDA AI/ML guidance (dataset documentation). Expected documentation: Training data source, curation methodology, annotation quality assurance, demographic/clinical composition. Current state: Model cards exist; full lineage not described in input. Recommended next step: Document training-data source and composition; confirm annotations meet clinical standard (single vs. consensus grading); retain documentation in DHF. Owner: Engineering + Regulatory Affairs. P2 FINDINGS (Best-practice improvements): (1) Labeling Update Since Clearance — FDA expectation: 510(k)-cleared labeling should be updated to reflect any algorithm updates, limitations, or new data. Current state: Unchanged since 2022. Recommended next step: Annual labeling review; update if post-clearance retraining occurred or new subgroup data available. Owner: Regulatory Affairs.