While the world debates AI agents taking over the web, India’s small publishers and regional content creators are already feeling the pain silently.
Imagine spending weeks writing a detailed article in Hindi carefully researched, verified, written for a reader in Meerut or Madurai only to find out that most of the “readers” who visited your page were not humans at all. They were bots. Automated programs scraping your content to feed into someone else’s AI engine, and sending zero ad revenue your way. This is no longer a hypothetical. For thousands of small Indian digital publishers, this is already their reality.
Earlier this month, Cloudflare, one of the world’s largest internet infrastructure companies, highlighted a milestone that many in the tech world had been anticipating: according to data observed across its network, bot and AI-generated web requests have now overtaken human-generated web requests for the first time. According to Cloudflare’s data, automated bot requests account for roughly 57% of all traffic to ordinary webpages, while humans account for just 43%. Cloudflare CEO Matthew Prince, who had predicted this crossover would happen in 2027, admitted he was caught off guard. “Welp, that happened faster than I predicted,” he wrote on X.
India is not watching from the sidelines
Most of the coverage around this shift has focused on American companies, Western publishers, and Silicon Valley executives. But India which has one of the largest and fastest-growing internet user bases in the world has as much at stake as anyone. In fact, arguably more.
India’s digital media sector overtook television as the country’s largest media segment for the first time in 2024, contributing 32% of total media and entertainment revenues. Digital advertising reached $8.18 billion, making up 55% of total ad spend. An enormous chunk of that money flows to smaller publishers, local news portals, regional-language websites, and independent blogs whose only currency is genuine human attention.
That currency is now being devalued quietly and rapidly. According to the 2026 Bad Bot Report by Thales, which analysed 17.2 trillion bot requests globally in 2025, bots already made up 53% of all web traffic worldwide, with 40% classified as “bad bots” designed to scrape data, commit fraud, or overwhelm servers. More alarmingly, AI-enabled bot attacks surged 12.5 times in a single year. These are not just cybersecurity statistics. For every bot that lands on an Indian publisher’s website and leaves without clicking an ad, there is a real rupee of revenue that simply evaporates.
The publisher squeeze: Traffic that pays nothing
Here is the cruel irony that small Indian publishers are now waking up to: their content is being read more than ever, just not by humans.
AI companies deploy bots called crawlers or fetchers that visit millions of pages to collect content, train large language models, or answer user queries in real time. According to a 2026 report by cybersecurity firm Akamai, AI bot activity surged 300% in 2025. The media industry which includes publishing companies ranked second globally in AI bot activity, with publishing organisations accounting for 40% of all AI bot traffic within that sector, making them the most heavily targeted sub-segment in one of the internet’s most attacked industries.
What does this mean in practical terms? When someone asks ChatGPT or Google’s AI Overview a question, an AI bot may have already scraped the answer from a news article written by a journalist in Lucknow or Pune without sending any traffic back to that journalist’s website. A 2026 Akamai report found that AI chatbots now drive approximately 96% less referral traffic than traditional Google Search.
Meanwhile, publishers across multiple markets have reported significant declines in referral traffic and advertising revenue as AI-generated answers increasingly reduce the need for users to click through to original websites. While the scale of the impact varies widely across organisations, concerns are growing that smaller publishers may be disproportionately affected because they depend heavily on search traffic and advertising revenue for survival.
Regional language content: The most vulnerable segment
If English-language publishers are struggling, the situation for India’s regional-language digital ecosystem is even more precarious.
Hindi, Tamil, Telugu, Bangla, and Marathi news portals typically operate on thin margins, smaller advertiser pools, and limited technical infrastructure. They rarely have the tools to detect bot traffic, let alone block it. Most rely entirely on Google AdSense or similar programmatic networks that automatically discount or exclude traffic flagged as non-human, meaning these publishers get paid less the moment bots start visiting their sites more.
It is a double punishment: the content gets consumed, but the creator earns nothing.
Meanwhile, the big players and large national news organisations with dedicated tech teams are at least somewhat equipped to fight back. They can implement bot-detection tools, negotiate licensing agreements with AI companies, or pivot to subscriptions. The small publisher in Nagpur running a local news website on a shared server cannot do any of that.
For them, the bot invasion is not a tech problem. It is an existential one.
According to industry research, AI companies are increasingly becoming some of the largest sources of automated traffic across the web. Akamai identified OpenAI as the largest source of AI bot traffic targeting media and publishing organisations, followed by Meta and ByteDance in second and third place respectively, with Anthropic and Perplexity rounding out the top five at lower volumes.
These are some of the most valuable technology companies in the world, and many publishers argue that their content is being used to train or power AI systems without a clear framework for compensation or attribution. While some large publishers have secured licensing agreements, smaller publishers often lack the leverage to negotiate similar arrangements.
Some publishers in the West have begun filing lawsuits. A handful of major news organisations have struck licensing deals with OpenAI and Google. But India has no equivalent policy conversation yet. There is no regulatory framework, no industry-wide negotiation, and no government mandate that requires AI companies to compensate Indian publishers for the content their bots consume.
While debates about AI regulation in India have largely centred around deepfakes and misinformation, the quieter crisis of AI bots hollowing out the economics of Indian digital publishing has barely registered.
The bot takeover of the internet is not going to reverse itself. AI agents are only going to get more capable and more widespread.
What can change is how India responds to it.
Indian digital publishers, especially the smaller, regional-language ones need awareness, tools, and policy support. Industry bodies such as the Digital News Publishers Association need to push for mandatory disclosure and compensation frameworks similar to those being discussed in parts of the European Union and Australia.
The Indian government, which has shown an appetite for regulating digital platforms, needs to bring AI scraping and bot traffic into that conversation.
The internet was always supposed to be a two-way street content creators publish, readers consume, and the system sustains itself. Bots have turned it into a one-way extraction machine.
Unless India’s publishers, policymakers, and readers wake up to this shift, a large part of India’s homegrown digital content ecosystem may quietly disappear scraped, summarised, and swallowed by AI systems that will never give credit where it is due.
Subscribe Deshwale on YouTube
