How to Use Synthetic Data to Build and Validate Models

Current as of October 30, 2026. This guide draws on US, UK and Canadian model risk guidance, the EU AI Act as amended in 2026, the EU General Data Protection Regulation, guidance and an opinion from UK and EU data protection authorities, NIST publications, and UK Financial Conduct Authority discussion papers, as published on that date. Which of them apply depends on the firm; it is not legal advice.

Synthetic data, generated by a model to resemble real records, lets teams train on rare events and test pipelines without exposing customer files. Among the model risk texts reviewed for this guide, only Canada's names it. OSFI's Guideline E-23, effective May 1, 2027, says development data "can also be synthetically generated or derived from proxy sources" and asks institutions to "implement controls to ensure quality and document provenance and use of synthetic data and proxy data."

This guide is for data science and validation teams at banks and insurers. Unless a step quotes a rule, it is a recommendation, labeled (interpretation), and that includes every fidelity, utility and privacy test not named in a text reviewed.

On scope: SR 26-2 (April 17, 2026), from the Federal Reserve, the OCC and the FDIC, supersedes SR 11-7 and SR 21-8 and is non-binding: it states that non-compliance "will not result in supervisory criticism", while its footnote 1 adds that "supervisory action may result for any violations of law or unsafe or unsound practices stemming from insufficient management of model risk." It is expected to be most relevant to larger banking organizations, and its footnote 3 says generative AI and agentic AI models "are not within the scope of this guidance." The PRA's SS1/23 (April 2026 version) applies to UK-incorporated banks, building societies and PRA-designated investment firms with internal model approval for regulatory capital; insurers are not in scope. OSFI's Guideline E-23, final since September 2025, will apply from May 1, 2027 to federally regulated financial institutions in Canada, banks and insurers included. The EU AI Act's data governance rules in Article 10 bind providers of high-risk AI systems; the high-risk list in Annex III includes creditworthiness evaluation and credit scoring of natural persons, except fraud detection, and risk assessment and pricing for natural persons in life and health insurance. Under Regulation (EU) 2026/1744, those obligations apply to Annex III systems from December 2, 2027. The ICO's guidance on privacy-enhancing technologies, which the ICO says is under review after the Data (Use and Access) Act, and the EDPB opinion cited below are regulator views, not legislation. The NIST publications are voluntary; SP 800-188 was written for US government agencies. The FCA's Feedback Statement FS23/1 (February 2023) and its Synthetic Data Expert Group's report (August 2025) are discussion material, not rules. The report says it "should not be taken as an indication of recommendations, guidance, or future policy" and "does not imply compliance with UK data protection law." It also says "The contents of this report do not represent the views of the FCA", nor those of any participating organization.

Step 1: State what the synthetic data is for, model by model

SR 26-2 says development "generally begins with a clear statement of purpose". The expert group report describes synthetic data used for training, as a substitute for testing or validation data, and for simulation and stress testing.

For each model, record the role synthetic data plays (augmenting training data, rebalancing rare classes, testing a pipeline or standing in for validation data), the synthetic share of each data set, and who approved it (interpretation).

Step 2: Treat the source data and the generator as personal data processing

In FS23/1 the FCA wrote that firms "will need to identify a lawful basis under GDPR" to use personal data both to generate synthetic data and to validate its utility. The PRA expects the development process to ensure "that usage of the data is compliant with data privacy and other relevant data regulations".

A generator trained on customer records is a model trained on personal data. The European Data Protection Board, in Opinion 28/2024 (adopted December 17, 2024), said "the EDPB considers that AI models trained with personal data cannot, in all cases, be considered anonymous." The opinion is about AI models and mentions synthetic data only in passing (paragraph 64, on data minimization, as other data that may make processing personal data unnecessary); applying its anonymity analysis to generators is our interpretation. Keep the generator under the same controls as its source data until shown otherwise (interpretation).

Step 3: Test whether the output is anonymous; do not assume it

The ICO warns: "Some synthetic data generation methods have been shown to be vulnerable to model inversion attacks, membership inference attacks and attribute disclosure risk." NIST SP 800-188 says "Fully synthetic datasets do not have a zero-disclosure risk".

The expert group report lists privacy tests its members use, including red teaming and membership inference attacks, and notes that "while synthetic data significantly reduces privacy risks, it may not eliminate them entirely." A practical privacy test set is: membership inference against the generator, distance from each synthetic record to its nearest real record, and checks that outliers have not been reproduced (interpretation). Record the conclusion, anonymous or still personal data, with its evidence (interpretation).

Step 4: Document provenance, generation and adjustments

OSFI expects development data to be "traceable (that is, having documented lineage and provenance)". The PRA's Principle 3.2(d) says: "Any adjustments made to the data used to develop the model or use of proxies to compensate for the lack of representative data should be clearly documented and subject to validation." The PRA text does not mention synthetic data; treating it as an adjustment or proxy under 3.2(d) is our interpretation.

For high-risk systems, Article 10(2) of the EU AI Act requires data governance practices covering, among other things, "data collection processes and the origin of data, and in the case of personal data, the original purpose of the data collection".

Record the source data and its date, the generation method, settings and seeds, any rebalancing or fairness adjustments and the generator version, and mark synthetic records in the data itself (interpretation).

Step 5: Test fidelity and utility against the intended use

FS23/1 described two kinds of measure: fidelity, "the statistical similarity of the synthetic dataset to the input real data", and utility, "evaluating its ability to meet a given use case or purpose". The FCA expert group report adds: "However, statistical similarity does not always equate to downstream performance."

For high-risk systems, Article 10(3) of the EU AI Act says data sets "shall be relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose."

A practical test set: compare distributions and correlations with the source; train on synthetic data and test on held-out real data, a method the expert group report describes; check rare events appear in realistic proportions; and compare feature importance between real-trained and synthetic-trained models (interpretation). Set pass criteria before testing, tied to the model's purpose (interpretation).

Step 6: Check for bias carried over or introduced

The ICO says "If you are generating synthetic data from personal information, any inherent biases in the information will be carried through." Under Article 10(2)(f) of the EU AI Act, data governance for high-risk data sets must include examination for possible biases.

Regulation (EU) 2026/1744 deleted Article 10(5) and replaced it with a new Article 4a carrying the same conditions. Under Article 4a(1), providers of high-risk systems may, where strictly necessary for bias detection and correction, exceptionally process special categories of personal data only if, among other conditions, "the bias detection and correction cannot be effectively fulfilled by processing other data", which the text says includes synthetic or anonymized data; the processing records must say why other data could not achieve that objective. Article 4a(2) extends the same basis, on the same conditions, to deployers of high-risk systems and to providers and deployers of other AI systems and models, and says "This paragraph does not create any obligation to conduct such bias detection and correction." Article 4a sits in Chapter I of the AI Act, which applies from February 2, 2025, so it has been available since the amending regulation entered into force in July 2026, ahead of the December 2, 2027 date for Annex III obligations.

Test fairness on both synthetic and real data, and document any fairness adjustment made during generation and its effect on the model (interpretation).

Step 7: Keep the decisive validation evidence on real data

SR 26-2 describes outcomes analysis as comparing "model outputs to corresponding real-world outcomes". NIST SP 800-188 says analysts "should be able to validate their discoveries against the original data".

Base the validation opinion on real held-out data wherever it exists, with synthetic-only results as support (interpretation). Where real data is scarce, record that as a limitation; the PRA expects non-representative development data to be reflected in the model's tier classification. Check for leakage between synthetic training data and real test data (interpretation).

If the generator is itself a generative AI model, SR 26-2 does not cover it, but the credit, fraud or pricing model trained on its output remains in scope (interpretation). Not every generator is generative AI in that sense. A statistical generator may itself meet a model definition, such as the PRA's "quantitative method, system, or approach" that processes input data into output, and then falls inside that framework (interpretation). NIST AI 600-1, a voluntary generative AI profile, warns, of generative models, that "Model collapse can occur when model training over-relies on synthetic data". Inventory the generator and regenerate when the source data changes materially (interpretation).

Checklist

Items restate the recommendations above (interpretation).

• The role and share of synthetic data is recorded for each model
• A lawful basis covers generation and utility testing on real data
• The generator is controlled like its source data until shown otherwise
• Privacy testing supports a recorded conclusion on whether the output is anonymous
• Source, method, settings, seeds, adjustments and generator version are documented, and synthetic records are marked
• Fidelity and utility pass criteria are set before testing, tied to the purpose
• Fairness is tested on synthetic and real data; adjustments are documented
• Validation rests on real held-out data, with limitations recorded
• The generator is inventoried and refreshed when its source data changes

How we sourced this guide: every factual statement in it is drawn from published rules, regulatory guidance and other public documents, and we keep a full set of sources and citations for it. If you would like them, or want to question a point, get in touch through our contact page and we will share them.