What the AI Feature Regulatory Risk Screen actually produces

DISCLAIMER: This is a directional pre-legal screen, NOT legal advice or a legal opinion. All flagged items require confirmation and interpretation by legal counsel before design or engineering work proceeds. --- EU AI ACT (Annex III, High-Risk Category) Trigger: Automated decision-making affecting employment outcomes (CV screening determines who advances to human review; low-ranked candidates are effectively filtered out without human touch). Employment is explicitly listed as high-risk under Annex III. Typical Obligations: Conformity assessment, human oversight requirements, transparency/explainability documentation, bias testing and monitoring, record-keeping of decisions, data quality standards. Key Question for Counsel: Does the feature's placement in the recruitment funnel (pre-human review, non-binding recommendation, or deterministic filtering) affect whether it is classified as high-risk or require conformity obligations? Consult counsel to confirm applicability and obligations. --- GDPR (EU) / UK GDPR Trigger: Article 22 (automated decision-making without human involvement): The system makes or significantly influences a decision (advancement to recruiter review) that produces legal or similarly significant effects on candidates. Article 6 (lawful basis): Processing CVs and inferred demographic signals require a valid lawful basis. Typical Obligations: Right to human review/intervention on automated decisions; transparency (privacy notices disclosing automated decision-making); lawful basis documentation; data subject rights (access, rectification, objection); possible impact assessments (DPIA) for high-risk processing. Key Question for Counsel: Does the feature's output constitute an "automated decision" under Article 22, or is it a recommendation that recruiters then decide on? If the latter, does it still require human review rights? Consult counsel to confirm applicability and obligations. --- NYC LOCAL LAW 144 (Automated Employment Decision Tools - AEDT) Trigger: Automated tool used to screen job candidates in NYC (one of the launch jurisdictions). Applies to most employers with 4+ employees. Typical Obligations: Pre-deployment bias audit by independent third party; annual recurrent bias audit; notice to candidates of use of AEDT before or at point of application; disclosure of material features and details of data used; record and transparency on performance metrics; right to opt out and human alternative process. Key Question for Counsel: What does "independent bias audit" entail, and what metrics/standards must the audit meet before launch? Does the feature's design require a human alternative workflow? Consult counsel to confirm applicability and obligations. --- ILLINOIS BIOMETRIC INFORMATION PRIVACY ACT (BIPA) Trigger: Feature processes CVs and infers demographic signals (e.g., age, gender, race from CV content or linked data). BIPA prohibits collection/retention of biometric information without consent and explicit authorization. "Demographic inferences" may qualify as biometric information in some interpretations. Typical Obligations: Explicit written consent before collection; disclosure of data use and retention; secure storage; ability for users to delete data; private right of action for violations (statutory damages $1,000–$5,000 per violation). Key Question for Counsel: Does the feature's inference of demographic signals from CV text and LinkedIn data trigger BIPA obligations, or does BIPA apply only to explicit biometric identifiers (fingerprint, facial recognition)? If CVs contain voluntarily self-reported diversity info, what consent/authorization is required? Consult counsel to confirm applicability and obligations. --- FTC AI GUIDANCE (US - Endorsement, Substantiation, Unfair/Deceptive Practices) Trigger: Feature claims to rank candidates by "job fit" or predict performance. FTC expects substantiation of performance claims and transparency about AI limitations. Deceptive use includes false claims of AI accuracy or failing to disclose AI involvement in a material decision. Typical Obligations: Substantiated claims about accuracy/fairness of the ranking; disclosure that candidates may be filtered by AI; transparency on what the model optimizes for and known limitations; documentation of testing and validation. Key Question for Counsel: What level of accuracy/bias testing is required to substantiate a "job fit" claim? Should the feature disclose to candidates that an automated system is screening them, and if so, at what point? Consult counsel to confirm applicability and obligations. --- EEOC AI GUIDANCE (US - Title VII, Adverse Impact and Disparate Treatment) Trigger: Automated screening model may have disparate impact on protected classes (race, color, religion, sex, national origin) if it correlates inferred or stated demographic signals with "job fit." Even facially neutral models can violate Title VII if they perpetuate historical hiring bias. Typical Obligations: Validation studies showing the model does not have adverse impact on protected groups; documentation of how the model was trained and tested; ability to demonstrate business necessity; monitoring for disparate treatment; records of candidate flow by protected class. Key Question for Counsel: What adverse impact analysis or validation study is needed before launch? Should the feature actively exclude demographic data from the model, or is it acceptable if the data is present but weighted as neutral? What data retention/analysis is required to monitor for post-launch disparate impact? Consult counsel to confirm applicability and obligations. --- COLORADO AI ACT (Section 12-5.7-1) Trigger: Feature is a "high-risk AI system" (Colorado's term for decision-making in employment, among other domains). Applies to systems deployed in Colorado or affecting Colorado residents. Typical Obligations: Human review or intervention option; notice that AI is being used; impact assessment; testing for bias and accuracy; opt-out or human alternative. Key Question for Counsel: What does "human review or intervention option" mean in practice—must every candidate get human review, or only those shortlisted by the model? What must the impact assessment and bias testing cover? Consult counsel to confirm applicability and obligations. --- UK INFORMATION COMMISSIONER'S OFFICE (ICO) AI GUIDANCE AND CODE OF PRACTICE Trigger: Feature processes UK residents' data for automated decision-making. ICO has issued guidance on AI and automated decision-making under UK GDPR; upcoming draft code of practice on AI and data protection provides non-binding but authoritative direction. Typical Obligations: Transparency on use of AI; impact assessment; fairness and lawfulness assessment; documentation of safeguards; human oversight and challenge mechanisms. Key Question for Counsel: What documentation and safeguards must the feature implement to comply with ICO expectations around algorithmic fairness and transparency in recruitment? Consult counsel to confirm applicability and obligations.

BEFORE LEGAL INTAKE: This feature implicates multiple overlapping regulations across three jurisdictions. Prioritize these questions with counsel in this order: 1. **Article 22 GDPR / UK GDPR (Automated Decision-Making):** - Does a "ranking" that leads to automatic filtering (low-ranked candidates not reviewed) constitute an automated decision under Article 22, or is it a decision-support tool where humans retain discretion? - If Article 22 applies, what specific human review, intervention, or opt-out rights must candidates have? 2. **NYC Local Law 144 (AEDT):** - Is this feature subject to NYC Local Law 144? (Confirm scope: applies to tools used to "screen" job candidates in NYC.) - What must the independent bias audit include, and when must it be completed before launch? - Must the feature offer candidates a human alternative process or opt-out? 3. **EU AI Act (High-Risk Classification):** - Is this feature classified as high-risk under Annex III (employment decisions)? - If high-risk, what are the conformity assessment, transparency, and human oversight obligations before and after launch? - Does the feature's design (pre-human review recommendation vs. deterministic filtering) affect the classification? 4. **Demographic Data & Bias (BIPA, EEOC, FTC):** - Does inferring or processing demographic signals (age, gender, race from CVs or linked data) trigger BIPA or other state biometric laws? - What adverse impact analysis or disparate impact testing is required under Title VII / EEOC guidance before launch? - Can the feature use demographic data as a feature in the model, or must it be excluded? 5. **Disclosure & Transparency (FTC, GDPR, NYC Local Law 144):** - At what point in the application flow must candidates be told that an automated system is screening them? - What information about the model (accuracy, features, data used, limitations) must be disclosed to candidates vs. to employers vs. to regulators? 6. **Data & Consent:** - What lawful basis under GDPR / UK GDPR justifies processing CVs and inferred demographic data? - If candidates voluntarily self-report diversity information, what consent or authorization is required (especially under BIPA)? - What data retention period is compliant, and what are deletion/opt-out obligations? 7. **Jurisdiction-Specific Gaps:** - For Illinois (BIPA) and Colorado (AI Act): Are there specific conformity, audit, or notice requirements we haven't identified? - Do any US states where the employer customers are headquartered impose additional requirements?

These pre-legal adjustments may reduce regulatory exposure, but counsel should confirm that each is material and sufficient for compliance. --- **CONSIDER: Human-in-the-Loop / Advisory Model** If the feature outputs a ranked list or recommendation but recruiters retain full discretion to review any candidate (including low-ranked ones) and make final decisions, this may move the system out of "automated decision-making" under Article 22 GDPR and reduce high-risk classification under the EU AI Act. Practically: design the recruiter interface so it is *easy* (one click) to review any candidate regardless of rank, and log all human decisions. Counsel to confirm whether this design change materially reduces obligations under Article 22 and EU AI Act high-risk requirements. --- **CONSIDER: Explicit Opt-Out & Human Review Path** Offer candidates an option to opt out of automated screening or request a human review of their CV before ranking. This may satisfy Article 22 GDPR rights and NYC Local Law 144 human-alternative requirements. Practically: add a checkbox at application asking candidates whether they consent to automated screening or prefer human review. Counsel to confirm whether opt-out alone satisfies Article 22 or if you must also guarantee human review for all candidates. --- **CONSIDER: Exclude Demographic Data From Model** Remove any demographic signals (inferred or stated: age, gender, race, national origin) from the ranking model features. Use only skills, experience, and role-fit signals. This may reduce BIPA exposure, disparate impact risk under Title VII, and fairness concerns under FTC/EEOC guidance. Practically: audit the feature's input data pipeline to strip demographic columns and ensure the model is not trained on proxy features that correlate with protected class. Counsel to confirm whether this design removes BIPA/EEOC risk or whether additional bias testing is still required. --- **CONSIDER: Transparent, Explainable Ranking** Design the recruiter UI to show *why* a candidate was ranked (e.g., "matched 8/10 required skills, 3 years relevant experience, X certifications"). This supports GDPR Article 22 transparency rights, EU AI Act explainability requirements, and FTC substantiation. Practically: log and display the top 3–5 factors that drove the ranking for each candidate. Counsel to confirm what level of explainability satisfies regulatory transparency obligations. --- **CONSIDER: Pre-Launch Independent Bias Audit** Commission a third-party bias audit before launch in any jurisdiction (especially NYC, EU) to test for disparate impact by protected class and validate accuracy claims. This demonstrates good-faith compliance with NYC Local Law 144 and EU AI Act conformity obligations, and provides defensible documentation if audited. Practically: partner with a vendor experienced in employment AI audits and collect disaggregated performance metrics (accuracy, false positive/negative rates) by race, gender, age, national origin. Counsel to confirm audit scope, metrics, and timing required. --- **CONSIDER: Candidate Disclosure & Notice** Include transparent language in the job posting and application confirming that CVs will be screened by an automated tool, explaining the role of the tool in the process (recommendation vs. filtering), and directing candidates to a page explaining what data is collected and how the ranking works. This addresses GDPR Article 13/14 transparency, NYC Local Law 144 notice requirements, and FTC deception concerns. Practically: add a brief disclosure in the application UI and link to a FAQ or explainer document. Counsel to confirm what information must be disclosed at point of application vs. on demand. --- **CONSIDER: Audit Logging & Candidate Data Rights** Design the system to log all CV screenings, ranking decisions, and recruiter actions (reviewed, contacted, rejected). This supports GDPR audit rights, demonstrates compliance with EU AI Act record-keeping, and provides data for EEOC/FTC investigation. Practically: create a candidate data export feature (GDPR Article 15 subject access request compliance) showing each candidate their ranking, the factors that drove it, and recruiter actions. Counsel to confirm what logs and exports are required by regulation. --- **CONSIDER: Jurisdiction-Specific Opt-In / Consent** If the feature will operate in multiple jurisdictions with different consent/notice requirements (EU GDPR, NYC, Colorado, Illinois), consider implementing jurisdiction-specific consent flows. For example: EU candidates may be shown a GDPR-compliant privacy notice and consent form; NYC and Colorado candidates may be shown NYC Local Law 144 and Colorado AI Act notices; Illinois candidates may be asked to consent to biometric processing (if demographic inference is used). Counsel to confirm whether a single global consent/notice model satisfies all jurisdictions or if differential treatment is required.