The European Data Protection Board's Guidelines 03/2026 on web scraping for generative AI are under public consultation. If your team trains models on scraped data, now's the time to prepare. These guidelines will clarify when web scraping counts as personal data processing under the GDPR, what legal basis applies, and how to handle data subject rights when your training corpus includes publicly accessible personal data.
This checklist provides your data governance team with a structured approach to audit your current web scraping practices and align them with the framework emerging from the consultation.
What This Checklist Covers
This checklist addresses GDPR compliance for web scraping operations that feed generative AI training pipelines. It's designed for teams that scrape websites, social media, forums, or other publicly accessible sources where personal data appears. You'll assess whether you're processing personal data, identify your legal basis, and confirm you've built the infrastructure to respond to data subject requests.
If you're relying on third-party datasets, this checklist still applies. You need to verify what your vendor scraped, how they did it, and whether they obtained valid consent or can demonstrate another legal basis.
Prerequisites
Before you start, gather these materials:
- Documentation of all web scraping operations (internal and vendor-provided)
- List of data sources (domains, platforms, API endpoints)
- Sample of scraped content showing what fields you're collecting
- Current data retention and deletion policies
- Records of any robots.txt exclusions or terms of service you've reviewed
- Your AI model's purpose specification and training documentation
You'll also need access to your legal team or DPO to confirm your legal basis determination.
Checklist Items
1. Identify whether your scraped data contains personal data
Review a representative sample of your scraped content. Personal data includes names, email addresses, usernames tied to individuals, IP addresses, profile descriptions, and any information that directly or indirectly identifies a person.
✓ Good looks like: You've documented which fields contain personal data and which don't. You can show that your sample size is statistically meaningful and that you've accounted for edge cases like user-generated content that might include unexpected personal identifiers.
2. Determine your legal basis for processing
If you're processing personal data, Article 6(1) GDPR requires a legal basis. For web scraping in AI training, legitimate interests (Article 6(1)(f)) is the most common claim, but you must conduct a balancing test. Consent is rarely viable at scale. Public task and contractual necessity won't apply to most commercial AI training.
✓ Good looks like: You've completed a documented legitimate interests assessment (LIA) that weighs your processing purpose against the data subject's reasonable expectations and fundamental rights. The LIA includes your consideration of whether the data was originally published with an expectation of further automated processing.
3. Respect robots.txt and terms of service
Check whether the sites you're scraping prohibit automated access via robots.txt or their terms of service. While these aren't GDPR requirements, violating them can undermine your legitimate interests claim and create separate legal exposure.
✓ Good looks like: You maintain a log of robots.txt files and terms of service for each domain you scrape. You've excluded domains that explicitly prohibit scraping, and you can demonstrate that you honor rate limits and access restrictions.
4. Implement data minimization in your scraping logic
Article 5(1)(c) GDPR requires that you collect only the personal data necessary for your specified purpose. If you're training a language model, do you need email addresses? Do you need full names, or would pseudonymized identifiers suffice?
✓ Good looks like: Your scraping scripts filter out unnecessary personal data fields before storage. You've documented why each retained field is necessary for your AI training objective. You're not scraping entire profiles when you only need text samples.
5. Build a mechanism to honor data subject rights
Data subjects can exercise their right to erasure (Article 17), right to object (Article 21), and right to rectification (Article 16) even if their data came from public sources. You need a process to identify and remove specific individuals' data from your training corpus.
✓ Good looks like: You've implemented a searchable index or hash table that maps scraped content back to its source and to any identifiable data subjects. You can locate and delete an individual's data within your GDPR response timeline (typically one month). You've tested this process with sample requests.
6. Assess whether re-training is required after deletion
If you delete personal data from your training set post-training, you face a choice: re-train the model without that data, or demonstrate that the data's influence on the model is negligible. Neither option is simple.
✓ Good looks like: You've documented your position on re-training obligations and consulted with your DPO. If you're not re-training, you can show that the deleted data represents a statistically insignificant portion of the training corpus and that the model doesn't reproduce identifiable information about that individual.
7. Draft transparent privacy notices
Article 13 GDPR requires that you inform data subjects when you collect their personal data, even from public sources. For web scraping, this often means publishing a notice on your website explaining your AI training practices.
✓ Good looks like: Your privacy notice specifies that you scrape publicly accessible data for AI training, names the categories of sources, explains your legal basis, and provides instructions for exercising data subject rights. The notice is easy to find and written in plain language.
8. Conduct a Data Protection Impact Assessment if required
Article 35 GDPR requires a DPIA for processing that's likely to result in high risk to individuals' rights and freedoms. Large-scale scraping of personal data for AI training often meets this threshold, especially if the AI system will make automated decisions or profile individuals.
✓ Good looks like: You've completed a DPIA that evaluates the risks of your scraping operation, identifies mitigation measures, and documents your DPO's review. If the residual risk is high, you've consulted your supervisory authority before proceeding.
Common Mistakes
Assuming public data is exempt from GDPR. It's not. GDPR applies to all personal data processing, regardless of whether the data was publicly accessible at the source.
Relying on implied consent. Posting data publicly doesn't constitute consent to scrape it for AI training. Consent under GDPR requires a clear affirmative action specific to your processing purpose.
Ignoring vendor due diligence. If you're using third-party datasets, you're still a controller (or joint controller). You can't outsource your GDPR compliance obligations.
Treating the training corpus as static. Data subject rights apply throughout the data lifecycle. You need ongoing processes, not one-time audits.
Next Steps
If you've identified gaps in this checklist, prioritize building your data subject rights infrastructure. That's the hardest piece to retrofit and the one most likely to surface in a regulatory inquiry.
Monitor the public consultation on Guidelines 03/2026. The EDPB will likely clarify whether certain scraping practices qualify for exemptions, how to interpret legitimate interests in the AI context, and what "reasonable expectations" means for data originally posted on social media or forums.
Finally, document everything. Your ability to demonstrate compliance matters as much as the compliance itself. When the guidelines finalize, you'll need to show your supervisory authority that you built your scraping operation with GDPR in mind from the start.
