What the QMS & Conformity Assessment Package actually produces

QMS STRUCTURE (13 elements per EU AI Act Article 17): (1) Regulatory Compliance Strategy — TalentSift's EU AI Act compliance program (launched Q1 2025) maps to this; reference existing ISO/IEC 42001:2023 AIMS and documented policy. Build: map Article 16 provider obligations to business functions; update compliance charter to reflect Annex III(4) employment context. (2) Design & Development Procedures — Reference ISO/IEC 42001 AI design processes and existing MLOps model-card process. Build: formalize bias audit protocol per ISO 42001 requirements (documented quarterly bias assessment per protected class). (3) Quality Control & Assurance — ISO/IEC 27001:2022 ISMS covers technical controls; ISO/IEC 42001 covers AI-specific QA. Build: checklist for pre-release bias testing (minimum 5% protected-class disaggregation) and explainability validation. (4) Verification & Validation — Existing IBM AI Fairness 360 audit (Q4 2024) demonstrates framework; formalize into recurring SOP. Build: specify test-case definition (e.g., at least 500 test cases per demographic cohort); version control for all changes. (5) Technical Documentation — Reference Annex IV checklist below. (6) Risk Management (Article 9 linkage) — ISO 14971 template exists; adapt for employment-discrimination risk (e.g., failure mode: algorithmic rank inversion for protected groups; mitigation: explainability + deployer override). (7) Post-Market Monitoring (Article 72) — Build: system design for collecting deployer feedback, adverse outcomes, and disputes; retention policy for 10 years. (8) Serious Incident Reporting (Article 73) — Build: escalation protocol for suspected discrimination incidents; notification timeline to authorities. (9) Authority Communication — Define primary contacts (Regulatory Affairs) and escalation path. (10) Record-Keeping — ISO/IEC 27001 document management system; add AI-specific records folder for model versions, audit logs, deployment registry. (11) Resource Management & Supply Chain Security — ISO/IEC 42001 team structure in place; formalize third-party risk assessment for cloud infrastructure (training, inference). (12) Accountability Framework — Define responsible functions: Chief Compliance Officer (overall accountability), Regulatory Affairs (documentation), Product Engineering (model governance), Risk (bias monitoring). (13) Notified Body Interaction — Not yet applicable; will be addressed if Annex VII route selected. EXISTING CONTROL LEVERAGE: ISO/IEC 42001:2023 certification is the largest asset (replaces need to build AI QMS from scratch); ISO/IEC 27001:2022 covers cybersecurity component of Article 15(2). Only gaps: formal employment-discrimination risk assessment, post-market monitoring infrastructure, and Article 73 serious-incident protocol.

ANNEX IV SECTIONS (9 required): (1) General description of the system — Product spec, intended use (screening support for HR teams), CV input scope, API integration points, versioning scheme. STATUS: Likely present (product roadmap exists). Build: formalize and version. (2) Design and development information, including training data spec and collection process — Existing model card partially covers; needs expansion on data-sourcing bias (gender/ethnicity distribution in training set, data-collection methodology). STATUS: Partial. Build: data-lineage document + quarterly representativeness audit. (3) Detailed description of the algorithms, architecture, and model type — Deep learning/transformer architecture? Explainability method? STATUS: Likely present. Build: formalize and version. (4) Information on testing performed including performance metrics — Existing IBM AI Fairness 360 audit (Q4 2024) provides baseline. STATUS: Partial. Build: expand to include disaggregated performance (accuracy by gender/age/ethnicity), fairness metrics (equalized odds, demographic parity), and edge-case testing. (5) Training and validation datasets — Sample size, composition, sourcing. STATUS: Likely present. Build: disaggregated composition report + consent/contractual basis for training data. (6) Assessment of model robustness and adherence to specifications — Adversarial testing (injection of biased signals), performance degradation under distribution shift. STATUS: Likely missing. Build: formal adversarial test suite + retraining frequency policy. (7) Risk management documentation per Article 9 — Failure modes, risk mitigation. STATUS: Likely missing for employment-discrimination context. Build: FMEA tailored to Annex III(4) risks (rank inversion by protected class, proxy discrimination via resume signal inference). (8) Monitoring and control mechanisms (Article 15) — Cybersecurity controls, drift detection, performance monitoring in production. STATUS: Partial (ISMS in place; AI drift monitoring may be missing). Build: real-time performance dashboard (accuracy, fairness metrics by deployer cohort) + cybersecurity incident-response plan. (9) Instructions for the deployer (Article 13 transparency + labeling) — User documentation on system limitations, bias disclosure, recommended human-review rate, feedback mechanisms. STATUS: Likely missing. Build: include labeling that explicitly states: system is decision-support only, does not make final hiring decisions, deployers must implement human review per Article 25, and candidate feedback mechanisms per GDPR.

FOR NOTIFIED BODY INTAKE (Annex VII route): (1) Data representativeness: What is the demographic composition (gender, age, ethnicity, national origin) of the training dataset, and does it reflect the customer base? Are validation studies powered to detect disparate impact by protected class? (2) Performance disaggregation: Can you provide false-negative and false-positive rates by demographic cohort (minimum 5 cohorts)? (3) Risk mitigation architecture: How does the system enforce human override, and what is the deployer's observed override rate by recommendation type? (4) Post-market monitoring: What real-world performance metrics will be collected, and over what frequency (daily/weekly/monthly)? (5) Cybersecurity controls: What access controls and audit logging protect the model and inference infrastructure per FDA/NIST AI RMF? FOR INTERNAL CONTROL REVIEW (Annex VI route): (1) Should we use internal or notified-body conformity assessment? Given employment-discrimination risk and regulatory scrutiny (EEOC, NYC Local Law 144), notified body is likely prudent. Confirm with external counsel. (2) QMS approval authority: who in TalentSift (quality function, CRO, legal) approves final Declaration of Conformity? FOR LEGAL COUNSEL: (1) Notified body selection: recommend contacting [name] (specializes in employment AI) or TÜV SÜD (Germany-based, covers EU AI Act) for Annex VII assessment. (2) Product liability insurance: does our D&O/E&O policy cover EU AI Act enforcement (fines up to €35M)? (3) Do our customer contracts assign Article 25 (human oversight, log retention) obligations clearly to deployers, and include indemnification if deployer breach causes GDPR/discrimination claims?