How to Test AI Underwriting Models for Unfair Discrimination

Current as of October 29, 2026. This guide draws on New York's Insurance Circular Letter No. 7 (2024), Colorado's SB21-169 and Regulation 10-1-1, and the Colorado Division of Insurance's 2023 draft testing regulation for life insurers, as published on that date. Which of them apply depends on the insurer, its products and its states; it is not legal advice.

Insurers using AI models and external data to underwrite or price policies in New York or Colorado are expected to show that those tools do not discriminate unfairly. Colorado's statute requires rules on how insurers demonstrate testing, and its governance regulation requires them to document that testing, but the Division of Insurance has not adopted the testing rule itself.

This guide sets out a testing program built around New York's expectations and designed to adapt when Colorado adopts a testing rule (interpretation). Unless a step quotes a rule, it is a recommendation, labeled (interpretation). Companion guides in this series cover vendor model validation and drift monitoring.

On scope: New York's Circular Letter No. 7, issued July 11, 2024, sets out the Department of Financial Services' expectations, as guidance, for insurers authorized in the state (with Article 43 corporations, HMOs, fraternal benefit societies and the State Insurance Fund) that use AI systems (AIS) and external consumer data (ECDIS) in underwriting and pricing. It "is not intended to address phases of the insurance product lifecycle other than underwriting and pricing". It does not apply to Child Health Plus, Essential Plan and Medicaid managed care coverage. Colorado's SB21-169 (C.R.S. 10-3-1104.9), enacted in 2021, is binding law that, as enacted, prohibits insurers from unfairly discriminating on race, color, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity or gender expression in any insurance practice; it excludes title insurance, surety bonds and most commercial policies. Regulation 10-1-1, binding and amended effective October 15, 2025, covers insurers offering individually issued life insurance, private passenger auto insurance and health benefit plans that use external consumer data and models that use it, in any insurance practice. Its framework is aimed at unfair discrimination "with respect to race".

Step 1: Inventory which AI models and data sources fall within each state's testing scope

New York defines AIS as a machine-based system designed to perform functions normally associated with human intelligence that is used to supplement or proxy for traditional underwriting or pricing, or to identify lifestyle indicators, and ECDIS excludes MIB exchange data, motor vehicle reports, prescription drug data and criminal history searches. In its summary of public comments, the Department said it intends to cover AIS whether or not they use external data. Colorado's regulation reaches external consumer data and the algorithms and predictive models that use it, and SB21-169 does not require life, annuity, long-term care or disability insurers to test individual medical, family history, occupational, disability or behavioral information with a direct relationship to mortality, morbidity or longevity risk "unless such information is otherwise included in the testing of an algorithm or predictive model that also uses external consumer data and information sources".

Tag each model with the states, lines and practices it touches and whether it uses external data, before running any statistics (interpretation).

Step 2: Estimate protected class membership when you do not collect it

Neither state's texts require insurers to collect protected characteristics. New York says testing should cover any protected class where membership "may be determined using data available to the insurer or may be reasonably inferred using accepted statistical methodologies", and that "There is no expectation that insurers collect additional data from, or about, individuals to perform exemplary analysis." A footnote gives, as an example, a Consumer Financial Protection Bureau report on the Bayesian Improved Surname Geocoding proxy method, and adds: "The Department is not endorsing any particular statistical methodology."

In its summary of comment themes, the Department added that the proxy provision "does not prevent insurers from using data to infer or impute protected class status for the limited purpose of the quantitative assessments".

Colorado's 2023 draft would have required "BIFSG and the insureds' or proposed insureds' name and geolocation information", with the categories "Hispanic, Black, Asian Pacific Islander (API),and White". Commenting on the draft, the American Academy of Actuaries warned that "various means of implementation can produce different results", citing the choice of census data and of single or probabilistic classification.

Pick one estimation method, record the census vintage and whether you use probabilities or assigned categories, and test sensitivity to that choice (interpretation).

Step 3: Measure disparities in approvals and premiums with more than one metric

New York says "insurers are encouraged to use multiple statistical metrics in evaluating data and model outputs to ensure a comprehensive understanding and assessment." The metrics it lists "may include, among others": adverse impact ratio, denials odds ratios, marginal effects, standardized mean differences, z-tests and t-tests, and drivers of disparity. If Step 1 of its assessment finds "no prima facie showing of a disproportionate adverse effect", the insurer "may conclude its evaluation after Step 1."

Colorado's draft, which has not been adopted, prescribed a specific test for life underwriting. Approval decisions were to be modeled by logistic regression, with a difference flagged "as indicated by a p-value of less than .05"; further testing followed only if the difference was "five (5) percentage points or greater as indicated by the marginal effects value of each BIFSG estimated race or ethnicity variable." Premiums per 1,000 of face amount were to be tested by linear regression, with further testing when a group's rate was at least 5% above the average for all policies. The Academy cautioned that "Use of p-values may not be indicative of bias with certain data samples."

Report effect sizes alongside significance tests, justify each control as risk-related, and treat the Colorado draft's method as a sensitivity check, not a safe harbor (interpretation).

Step 4: Test external data and model variables for proxy effects

New York's proxy assessment says "Insurers should evaluate the extent to which such ECDIS are correlated with ( i.e. , proxy for) status in any protected classes that may result in unfair or unlawful discrimination." If correlations are found, insurers "should consider whether the use of such ECDIS is required by a legitimate business necessity."

SB21-169 defines unfair discrimination to include data or models correlated with a protected class whose use results in a disproportionately negative outcome "which negative outcome exceeds the reasonable correlation to the underlying insurance practice, including losses and costs for underwriting." Colorado's draft would have compared each external variable's coefficient in models run with and without race and ethnicity terms, and "If any ECDIS variable exhibits a difference, that ECDIS variable, and the algorithm or predictive model that uses that ECDIS variable, is deemed to have a direct relationship to a disproportionate negative outcome and is thus unfairly discriminatory requiring remediation." The Academy's comment letter objected that "Adding variables to a regression often causes coefficients for variables already in the model to change, because regression coefficients are jointly estimated."

When an outcome test flags a gap, run variable-level attribution and record each variable's predictive value beside its disparity contribution; read literally, the draft's coefficient test would flag many variables (interpretation).

Step 5: Document the rationale, then search for less discriminatory alternatives

If a disproportionate effect appears, New York's Step 2 asks whether there is "a legitimate, lawful, and fair explanation or rationale for the differential effect on similarly situated insureds"; without one, the insurer should modify its use and retest from Step 1. Step 3 is "conducting and appropriately documenting a search and analysis for a less discriminatory alternative variable(s) or methodology that would reasonably meet the insurer's legitimate business needs." New York adds: "If no less discriminatory alternative exists, the insurer should conduct ongoing model risk management consistent with Section III of this guidance, and repeat Step 3 at least annually."

Record each candidate tried with its accuracy and disparity results (interpretation).

Step 6: Add a qualitative review that explains how the model works

New York expects the comprehensive assessment to "include a qualitative assessment of unfair or unlawful discrimination", which "includes being able to explain, at all times, how the insurer's AIS operates and to articulate a logical relationship between ECDIS and other model variables with an insured or potential insured individual's risk."

Have actuarial, underwriting and compliance reviewers sign a memo per model giving each external variable's risk rationale (interpretation).

Step 7: Set the testing cadence and remediation triggers

New York says "Unfair or unlawful discrimination testing and analysis should be administered prior to putting AIS into production and on a regular cadence thereafter, as well as whenever material updates or changes are made to either the ECDIS or AIS." SB21-169 requires Colorado's rules to give insurers "a reasonable period of time" to remedy an unfairly discriminatory impact.

Test before launch, at least annually and on every material model or data change, and route flagged results into the revalidation process in the companion guide (interpretation).

Step 8: Keep testing records ready for the regulator

New York says "An insurer should be prepared to make such documentation available to the Department upon request." In Colorado, life insurers that use external consumer data, or models that use it, must file annual compliance reports from December 1, 2024, and auto and health benefit plan insurers that do so from July 1, 2026; insurers that use none file an annual officer attestation instead. Each report must be signed by an officer attesting to compliance, or come with a corrective action plan. Keep methods, data, code versions, results and decisions per model and test cycle (interpretation).

Checklist

Items restate the recommendations above (interpretation).

• Models are tagged by state, line, practice and external data use
• The class estimation method is documented and stress-tested
• Approvals and premiums are tested with several metrics and effect sizes
• Flagged gaps get variable-level attribution
• Each flagged disparity has a documented rationale and alternatives search
• A qualitative memo explains each external variable's link to risk
• Tests run before launch, at least annually and after material changes
• Colorado reports and attestations are calendared, and the docket is watched for a testing rule

How we sourced this guide: every factual statement in it is drawn from published rules, regulatory guidance and other public documents, and we keep a full set of sources and citations for it. If you would like them, or want to question a point, get in touch through our contact page and we will share them.