Skip to content
ScrapeNest
All posts
·7 min read·by ScrapeNest Team

The SerpApi ruling, and why the robots.txt answer flips in Europe

A US court just gutted Google's DMCA case against SerpApi, and the coverage is getting the reasoning wrong. Here is what the order actually holds, and why the EDPB is moving in the opposite direction on the exact same question.

legalweb-scrapingdmcagdprserp-scraping

On July 20, Chief U.S. District Judge Yvonne Gonzalez Rogers threw out most of Google's DMCA case against SerpApi. The coverage has been enthusiastic and, in places, wrong about why. Since we get asked about this weekly, here is what the order actually says, and what it does not do for anyone operating out of the EU.

What the court held

Google sued in December 2025, claiming SerpApi got past SearchGuard, its anti-scraping system, to collect and resell search results, in violation of the DMCA's anti-circumvention rules (17 U.S.C. § 1201). SerpApi moved to dismiss in February.

The part people are getting wrong: the judge did not rule that SerpApi's methods aren't circumvention. Spoofing browser fingerprints, rotating IPs and solving CAPTCHAs to get past SearchGuard is circumvention. It simply isn't unlawful on its own. Section 1201 only bites when the measure effectively controls access to a copyrighted work and is deployed with the copyright owner's authority. On plain organic results there is no copyrighted work behind the wall, so there is no violation. That portion is dismissed for good, and Google can't refile it.

Where a result does contain a copyrighted component, typically a licensed image in a Knowledge Panel, the claim survives in principle. Google just didn't plead that it deployed SearchGuard with the relevant owner's authorisation. It has 21 days to fix that. Discovery is stayed meanwhile.

Google won one point worth noting: SerpApi argued it couldn't be sued at all because Google isn't the copyright holder of the third-party content at issue. The court disagreed. Section 1201 claims aren't limited to copyright owners.

SerpApi isn't out of the woods either way. Reddit filed a similar DMCA suit in October 2025 against SerpApi and others, including Perplexity and Oxylabs, over Reddit content pulled from Google results.

The line courts are drawing

It runs between protecting access and protecting a copyrighted work. A bot-blocker in front of prices, listings, job postings or organic results guards nothing the DMCA was written to guard.

Ziff Davis v. OpenAI went the same way on robots.txt, with a New York federal court holding that robots.txt files, as pleaded there, are not technological protection measures, closer to a "keep off the grass" sign than a lock. The court was careful to frame that as specific to the case in front of it, so don't file it as settled doctrine.

People reach for hiQ v. LinkedIn here, but hiQ was a CFAA case about whether scraping public profiles is access "without authorization." It never touched the copyright question. And it's a cautionary tale rather than a victory lap: hiQ won the statutory argument, lost on breach of contract, and folded. Winning the statute is not winning the case.

In Europe, the robots.txt answer flips

None of this binds anyone here. The DMCA is a US statute, and no tribunal judiciaire is going to cite it.

More to the point, the EU is moving in the opposite direction on the exact question Ziff Davis answered. On 7 July the EDPB adopted Guidelines 03/2026 on web scraping in the context of generative AI, now open for consultation. Three things in there matter:

Consent is not a realistic legal basis for scraping at scale, so legitimate interest under Article 6(1)(f) is the avenue, which means everything rides on the balancing test. And in that test, the EDPB treats robots.txt, ai.txt, CAPTCHAs and login walls as signals of what data subjects reasonably expect.

So the same act carries opposite weight depending on where you stand. In the US, robots.txt is a sign you may legally walk past. In the EU, walking past it is evidence against you whenever personal data is in scope. Ignoring access controls stopped being purely a copyright or terms-of-service question here. Guidelines aren't binding and the text may shift in consultation, but the direction is not ambiguous.

The other European framework worth knowing is the sui generis database right under Directive 96/9/EC, which protects the investment in compiling a database regardless of whether any individual data point is copyrighted. That is much closer to what Google was reaching for, and it's what an EU rights holder would actually plead. The Data Act, applicable since September 2025, sits alongside it for data generated by connected products and services.

What still bites

Contract, mostly. Accept terms prohibiting automated access and the DMCA question stops mattering, as hiQ discovered. CFAA claims survive but courts have narrowed them toward genuine credential misuse. Section 1202 CMI claims, about stripped metadata and author names, are getting real traction in AI litigation and are a separate question entirely from circumvention.

And provenance. In Bartz v. Anthropic, authors sued over the books used in training. The June 2025 ruling found training on lawfully purchased books was fair use while building a permanent library from pirated sources was not, and Anthropic paid $1.5 billion to settle the piracy claims. A settlement sets no precedent, but the ruling behind it tells you where the line sits: how data was obtained and whether you kept it matters as much as what you did with it. Anthropic's problem was downloading from pirate libraries, not scraping.

How this shapes what we build

We host in the EU because routing through a US server and hoping isn't a compliance posture. We keep publicly accessible non-personal data (prices, availability, rankings, listings) on a different track from anything touching personal data or licensed media, because the analysis genuinely differs. We're deliberate about rate limits, since degrading someone's service invites claims that have nothing to do with copyright. And we watch the EU side closely, because that's where our customers' exposure actually lives.

The SerpApi order is a good result and a narrow one. It tells you how far a platform can stretch a copyright statute to cover its bot-blocker. It doesn't tell you anything about your GDPR position.

Sources

The SerpApi order - Google LLC v. SerpApi LLC, No. 4:25-cv-10826-YGR (N.D. Cal.), complaint filed 19 December 2025, order granting the motion to dismiss issued 20 July 2026.

robots.txt in the US - In re OpenAI, Inc., Copyright Infringement Litigation (S.D.N.Y.), opinion of 15 December 2025 on the Ziff Davis claims.

hiQ v. LinkedIn - hiQ Labs, Inc. v. LinkedIn Corp. (N.D. Cal. / 9th Cir.).

The EU position

Provenance and training data - Bartz v. Anthropic PBC, No. 3:24-cv-05417 (N.D. Cal.), fair use order of 24 June 2025.

This article is provided for informational purposes and does not constitute legal advice. If you have questions about the legality of a specific scraping use case, consult a lawyer familiar with data and IP law in your jurisdiction.

About ScrapeNest

ScrapeNest is a French scraping platform for teams that need reliable, compliant extraction at scale. We run entirely inside the EU, on infrastructure built for GDPR, with people who follow European data law closely enough to give you a straight answer.