The Great AI Licensing Chill: Why Publishers Are Shifting from Bulk Deals to Pay-Per-Answer Models

The relationship between the world’s major media publishers and the artificial intelligence giants currently building the future of the internet is undergoing a profound and somewhat contradictory shift. For the past two years, the industry narrative was dominated by the pursuit of massive, multi-year licensing agreements. Companies like OpenAI, Meta, and Amazon were viewed as the new financial engines for journalism, offering significant payouts in exchange for the right to ingest vast swathes of human-written content to train their Large Language Models (LLMs).

However, as we move deeper into 2025, that initial gold rush appears to be losing its momentum. The once-frenetic pace of deal-making has slowed to a crawl, particularly for those publishers who have spent months—or even years—waiting in the wings for a contract that may now never arrive.

The Market for Licensing Deals Stalls

The signs of a cooling market are increasingly visible. At a recent London town hall hosted by Digiday, industry executives openly discussed a palpable shift in the air: the conversations with AI model makers, once characterized by urgency and high-stakes negotiation, have noticeably quieted.

This sentiment is echoed by experts who track the industry’s digital infrastructure. Katya Gorchinskaya, a journalist and media consultant who spearheaded the development of J·Index—a comprehensive database tracking thousands of AI implementations within newsrooms globally—recently noted in an interview with the Reuters Institute for the Study of Journalism that the initial window for blanket licensing deals appears to be effectively closing. For many mid-sized or independent publishers, the dream of a massive, upfront payment from an AI company to "unlock" their archives is becoming increasingly elusive.

This stagnation presents a paradox. While the appetite for direct licensing deals among AI companies seems to be waning, the actual demand for content—the raw, human-authored data required to keep these models relevant and accurate—is surging.

The Bot War: Scraping at Unprecedented Scales

Even as licensing deals stall, the mechanical strain on publisher websites has never been higher. The internet is currently awash in automated agents, with AI companies and third-party scrapers sending waves of bots to ingest information at a scale that challenges the technical infrastructure of even the largest media houses.

I recently explored this issue with Jonathan Roberts, the chief innovation officer at People Inc., for The Media Copilot podcast. Roberts provided a startling window into the sheer volume of this activity. According to his data, the number of distinct, unauthorized bots attempting to scrape content from media properties like People, InStyle, and Investopedia has reached the tens of thousands on a daily basis. To manage this, the company has adopted a strict policy: they block almost every request, allowing only 38 verified bots to proceed.

The technical toll of this scraping is not merely a theoretical annoyance; it is a significant cost center for publishers. During the same Digiday event in London, Fiona Spooner, managing director of consumer revenue at the Financial Times, shared a staggering anecdote about the scale of these attempts. She noted that a single trial account, leveraging roughly 700,000 different IP addresses, attempted to access and ingest nearly one million articles in the span of just eight hours.

For many media outlets, the natural response to this "bot-hammering" is to treat it as an existential threat. When a site is forced to devote its server capacity to managing millions of unauthorized requests, the immediate business decision is to block them entirely to save on infrastructure costs and protect intellectual property. However, this defensive posture creates a long-term strategic dilemma: if a publisher’s ultimate goal is to participate in the AI ecosystem and monetize their content, an "all-or-nothing" approach—blocking everything—offers little leverage to secure a seat at the table.

Shifting Toward Metered Pricing

As the initial era of "bulk" licensing agreements fades, the industry is pivoting toward a more granular, transactional approach. We are seeing a slow but steady evolution toward metered pricing, a model that promises to bring order to the chaos of automated scraping. While the concept of metered access is not new—it has been a staple of the digital subscription economy for years—the question now is how to apply it effectively to AI, and specifically, what exactly the "meter" should measure.

Early iterations of this model, such as those introduced by TollBit, focused on a "pay-per-crawl" structure. This allowed publishers to direct automated bots to a digital gate, where the software would either pay a set fee to access the content or be denied entry. The company reports that this technology is now deployed on more than 7,000 publisher sites. Cloudflare similarly entered the fray last year with a high-profile announcement that it would block AI bots by default for new sites on its network, simultaneously offering publishers the tools to charge entities for every crawl.

Yet, the industry is already moving past the "per-crawl" mentality. The prevailing wisdom now leans toward a "pay-per-use" model. In this framework, the value is not extracted when a bot scrapes a page, but rather when the information contained on that page is actually utilized to generate an answer for an end user.

This shift marks a more sophisticated attempt to align revenue with the utility of the content. Cloudflare, for instance, pivoted its program in July, moving away from a flat "per-crawl" fee to a structure where publishers are compensated when their content contributes to a response. Partnerships with entities like Ceramic.ai and You.com are serving as the initial test cases for this transition. Similarly, major players like Google and Microsoft are currently exploring or piloting their own versions of these value-based licensing programs.

While these programs remain in their beta stages, they represent a significant shift in the power dynamic. Lara Cohen, Cloudflare’s VP of strategic partnerships for media, creators, and AI, has indicated that the company is actively working toward a more robust system. The goal is to create a dynamic marketplace where publishers are alerted in real-time when an AI buyer seeks to use their content, providing the publisher with the agency to accept, decline, or negotiate the terms of that specific interaction.

The transition from bulk licensing to a transactional, usage-based model signals that both publishers and AI developers are still feeling their way toward a sustainable equilibrium. For publishers, the era of expecting a singular, massive windfall from AI companies is likely drawing to a close. In its place, a more complex, high-volume, and data-driven marketplace is emerging—one where the value of a piece of journalism is no longer measured by the total size of an archive, but by its specific relevance and contribution to the AI-generated answers of tomorrow. Whether this new model will generate the revenue necessary to sustain high-quality newsrooms remains the defining question for the industry in the coming years.