Artificial intelligence is changing how UK businesses collect, analyse and use personal data. Tools that summarise customer correspondence, screen applications or support business decisions can introduce processing that existing privacy documentation does not cover. The UK GDPR and Data Protection Act 2018 remain central, alongside changes introduced by the Data (Use and Access) Act 2025.
AI and data protection risk assessments need to reflect the actual system, its purpose and its impact on individuals. Herrington Carmichael advises businesses on the legal implications of AI adoption. This article explains where assessments may need to change and how organisations can document a defensible approach.
Why Traditional DPIAs May No Longer Be Enough
A static Data Protection Impact Assessment may miss AI-specific risks, although the assessment itself remains an important compliance tool. A DPIA is required where processing is likely to result in high risk to individuals’ rights and freedoms. AI use should be assessed individually; not every deployment automatically requires a DPIA.
The ICO’s AI guidance explains that assessments should consider data flows, necessity, proportionality and potential harm. An existing assessment for conventional software may not be sufficient where AI functionality changes how personal data is processed. For example, an AI tool may profile individuals, use information to train or improve models, or share data with additional service providers that were not involved in the original processing. These changes can introduce new privacy and compliance risks that were not considered in the original assessment.
Businesses should therefore assess AI systems before processing begins and review their DPIAs whenever an AI tool’s purpose, functionality or risk profile changes. Vendor documentation can support this process, but it cannot replace an assessment of how the organisation itself intends to use the technology.
How AI Systems Process Personal Data
AI can process personal data through inputs, analysis, training and outputs. A prompt may contain customer details, whilst an uploaded document may include employee information that is unnecessary for the intended task.
Generated summaries and predictions may also constitute personal data where they relate to identifiable individuals. An AI system could infer health information from apparently ordinary records, requiring consideration of special category data rules as well as a lawful basis.
Organisations should distinguish between using an AI tool to assist with a task and allowing information entered into that tool to be used for model training or service improvement. Permission to process information for one purpose does not automatically justify its reuse for another.
Businesses should also consider the risks associated with confidential information and intellectual property. Employees may inadvertently upload commercially sensitive material, client data, draft contracts, source code, business strategies or other proprietary information into publicly available AI platforms. In some cases, organisations have faced legal and regulatory concerns where confidential information was input into AI systems without appropriate safeguards. Assessments should therefore consider what information may be shared with AI tools, whether it may be retained or used by providers, and what contractual and technical controls are required to protect confidentiality and intellectual property rights.
Identifying Hidden Data Protection Risks
AI risks can arise from informal staff use, inaccurate outputs and unexpected inferences. Employees using unapproved tools may disclose personal information outside established procurement and security controls.
For example, an AI tool used to draft reports or analyse documents could inadvertently reproduce confidential information in its outputs or create uncertainty about ownership of AI-generated content. Assessments should consider both the impact on individuals and the potential commercial consequences of disclosing sensitive information, losing control of intellectual property or breaching contractual confidentiality obligations. Internal policies should provide clear guidance on the acceptable use of AI tools and the types of information that must not be entered into public systems.
Organisations should also test whether information can be located, corrected or deleted when individuals exercise their rights. Apparent anonymity should not be assumed merely because names have been removed.
Vendor Due Diligence for AI Platforms
AI vendor due diligence should establish what the provider does with personal data and which responsibilities each party holds. A supplier may act as a processor for one activity and a controller for another, depending on its actual purposes and decisions.
Businesses should examine retention, training settings, subprocessors, security evidence and assistance with individuals’ rights. Where a processor is used, the agreement must meet applicable UK GDPR requirements.
Commercial contract reviews should also address changes to services, incident reporting and exit arrangements. A statement that a platform is “GDPR compliant” does not establish that a particular deployment is lawful.
International Data Flows and AI Training Models
Overseas AI processing requires an assessment of applicable international transfer rules. Businesses should identify where information is stored, accessed and shared, including support access and subprocessors, rather than relying only on the advertised hosting location.
Where a restricted transfer occurs, an appropriate route is needed, such as applicable adequacy regulations or suitable contractual safeguards and the required assessment. Vendor reuse for training must also be assessed for lawfulness and purpose compatibility. These questions matter when procuring tools and when investing in AI businesses. Contractual assurances should match the processing that takes place.
Governance and Board Responsibilities
Senior management should oversee AI data protection risks. Technical teams and vendors cannot take over that responsibility.
Boards should receive understandable information about higher-risk uses, unresolved concerns and controls. DPO involvement, where applicable, should begin early, supported by legal, procurement and operational colleagues.
The Data (Use and Access) Act 2025 changed the automated decision-making framework. Significant solely automated decisions require safeguards, including opportunities for human intervention and challenge; special category data attracts additional restrictions. Human review should be meaningful, with authority to reconsider outcomes.
Building a Defensible AI Data Protection Framework
A defensible framework records why AI processing is justified, how risks are controlled and when decisions will be reviewed. Organisations should consider:
- Maintaining an inventory of approved AI uses.
- Mapping data flows and documenting purposes, lawful bases and relevant additional conditions.
- Screening for DPIAs and assessing less intrusive alternatives.
- Recording risks to individuals, safeguards, evidence and residual risk.
- Updating privacy notices, staff policies and rights-handling procedures.
- Setting review triggers for new features, vendors, incidents or purposes.
Where a required DPIA identifies high risk that cannot be sufficiently reduced, the organisation must consult the ICO before processing starts. Herrington Carmichael’s AI Readiness Audit provides a structured legal review of current or proposed AI use.
Key Questions Businesses Should Be Asking in 2026
Does every AI tool need a DPIA?
No. A DPIA is required where processing is likely to create high risk to individuals. Businesses should document their screening decision and assess the specific purpose, information involved and potential effects.
Can customer data be used to train AI?
Only where the processing meets applicable legal requirements. For example, businesses should assess lawful basis, purpose compatibility, transparency and contractual restrictions rather than assuming existing access permits training.
When should an AI risk assessment be reviewed?
Review it when changes could affect processing or risk, including new purposes, information, model functionality or recipients. Monitoring should establish whether safeguards remain effective in practice.
How We Can Help
Herrington Carmichael’s Data Protection team advises on compliance, data sharing and international transfers. Our Commercial & Regulatory team support businesses with AI-related supplier terms, privacy policies and governance arrangements.
To discuss your organisation’s AI use, vendor contracts or data protection framework, contact us to speak with our team.
