How to track competitor pricing page changes without false alerts

How to track competitor pricing page changes

Tracking a competitor's pricing page means re-reading the same short list of URLs on a schedule and comparing named fields between visits: the price strings, the plan names, the headline and the buttons. Comparing the whole page does not work, because a modern pricing page changes some of its bytes on most loads. In the test described below, the raw HTML of two public pricing pages differed between consecutive daily captures 211 times out of 250 in 2025. The prices on those pages differed 18 times, and only 5 of those 18 were in the same currency on both sides.

The method that holds up has five steps:

1. Pick five to eight pages per competitor where a change would lead to a decision.
2. Re-read them on a fixed schedule, no more than once a day, after reading the site's robots.txt.
3. Extract the fields you care about from each reading: title, main headline, button labels, and each price with its plan name and billing period.
4. Compare those fields with the volatile tokens masked, and label currency switches, billing-period switches and returning variants separately from real changes.
5. Keep a record of every visit, including the ones that failed, and show each change as the text before and the text after.

The sections below cover which pages to pick, how often to read them, what the measured noise looks like, how to filter it, how to read a change, and what an empty change list does and does not mean.

Which competitor pages are worth watching

The competitor pages worth watching are the ones where a change means a decision for you: the pricing page, the homepage, the landing pages their ads point to, any page that compares them with you, and the signup page. Each one answers a different question:

• The pricing or plans page shows plan prices, plan names, what each plan includes and whether a free tier or trial exists.
• The homepage headline and main button show what the competitor is leading with this quarter.
• Ad landing pages show the offer they are paying to promote. An ad library shows the link in each ad, as our guide to what each ad library shows about a competitor explains.
• Comparison and alternatives pages that name you show how they position against you.
• The signup page shows trial length, whether a card is required and which plan a new account starts on.

Pin the exact URLs you mean rather than letting a tool guess them. A tool that guesses a /pricing address and gets a 404 has learned nothing about the competitor. A page you pinned that starts returning 404 is news: the page was moved or removed.

How often to check a competitor's pricing page

Once a day is the most often a competitor's pricing page is worth reading, and once a week is enough for most teams. The right cadence is the speed at which you would act on a change. In the test below, the one plan price move stayed put: the $20 Business price that appeared on Notion's page between May 12 and May 16, 2025 was on every later dollar capture through December 19, 2025, so a weekly check would have found it within a week.

Reading another company's site carries rules, set out in two IETF standards and in Google's crawler documentation:

• RFC 9309, the IETF standard for robots.txt published in September 2022, says a crawler may fetch anything when robots.txt answers with a 4xx status, and must assume everything is disallowed when it answers with a 5xx status or a network error. It also says a cached copy of robots.txt should not be used for more than 24 hours, which matches a daily read.
• Crawl-delay is not part of RFC 9309, and Google's robots.txt documentation says its crawlers do not support the field. Honouring it anyway costs little when the job is a handful of pages a day.
• RFC 6585 defines status 429, Too Many Requests, and says the response may carry a Retry-After header saying how long to wait. A polite crawler stops the run and waits rather than retrying straight away.

Some pricing pages cannot be read without a browser. Google's JavaScript guide describes the app shell model, where the first HTML response does not contain the page's content and Google renders it later in a headless Chromium. A tracker that reads HTML without running JavaScript sees an empty page there, and it should say so instead of reporting that the prices disappeared.

Why comparing whole pages alerts on almost every visit

Comparing whole pages alerts on almost every visit because pricing pages change their markup, their scripts and their rotating copy far more often than they change their prices. To measure how much more often, we took every 2025 capture of two public pricing pages from the Internet Archive's Wayback Machine, one capture per day, from the Notion pricing page index and the Zapier pricing page index. The method, run on October 18, 2026:

1. List the year's captures with the archive's CDX API, keeping successful responses and one capture per day.
2. Fetch each capture exactly as archived, without the archive's own toolbar or rewritten links, and decompress it.
3. Compare each capture with the one before it four ways: the raw HTML; the visible text, with scripts, styles and markup removed; the visible text with volatile tokens masked (the masks are in the next section); and the set of price strings, such as $15 or €9.50.

Three readings follow from the table. First, a raw comparison tells you almost nothing: it fired on 211 of 250 pairs. Second, visible text is better but still noisy, and part of that noise is variants rather than edits. On Notion's page, 15 of the 51 text changes returned the page to a version seen within the previous five captures, and one paragraph about offline mode was added and removed three times each between the captures of August 16 and September 1, 2025. Third, even price strings need labelling: 7 of Notion's 118 captures were a euro version of the page, which produced 13 of its 14 price differences.

The test has limits worth stating. The archive captures a page when it chooses to, so 118 captures cover 118 of the 351 days between January 3 and December 19. The archive also fetches from its own locations, which is the likely source of the euro captures. And Zapier's page splits its cents into a separate element, so our simple price pattern read $19.99 as $19; the split was the same on every capture, so it did not change the counts.

How to filter out timers, counters and A/B tests

Filtering a competitor page for noise means masking the tokens that change on every visit, treating a version you have seen before as a variant, and giving currency and billing-period switches labels of their own. Each source of noise has its own filter:

The variant rule rests on how sites run experiments. Google's A/B testing guidance describes tests that redirect some visitors to variation URLs and tests that insert variations on the same URL with JavaScript, so two readings of one address can legitimately differ without the competitor having changed anything. The currency rule rests on how prices are localised: Stripe's Adaptive Pricing, for example, shows customers prices in their local currency, chosen by location, in more than 150 countries.

The masks in the table are the ones overads' own crawler uses, read from its code on October 18, 2026. That code treats its thresholds as starting guesses to be tuned on real pages: it calls a page a variant when its current version matches one of its last five, and it stops reporting body-text changes on a page whose body changed on more than five visits in a row while staying at least 90 percent similar.

How to read a detected price change

A detected price change should be read as two strings, the text before and the text after, with the plan name and billing period beside each, and never as a percentage a tool worked out on its own. Notion's 2025 move is a clean example. The May 12, 2025 capture lists Business at $15 per seat a month, and the May 16, 2025 capture lists Business at $20 per seat a month. The plan, the currency and the period match on both sides, so this is a real move.

Four other shapes look like a price change and are not one:

• The same plans in a different currency are a currency switch, usually caused by where the page was read from.
• The same amounts with a different period, such as per month becoming per year, are a toggle default flipping.
• Prices on one side and none on the other mean one reading found no prices in the HTML, not that the competitor removed them.
• A change that reverses within a few readings is usually a test variant. Wait for the next reading before acting on it.

A percentage is only meaningful when both amounts parse as numbers, in the same currency, for the same plan and the same period. Even then, open both versions of the page before telling anyone: the before and after strings are the evidence, and the percentage is a summary of them.

What an empty change list does and does not mean

An empty change list means a competitor's page did not change only when the page was read and compared; five other situations produce the same empty list. A tracker that shows them all as no changes is reporting its own failures as facts about the competitor.

overads' API makes this distinction part of every answer. Its analytics reference documents a provenance.emptyMeans field on empty results, with three values: no_source_connected, collection_has_not_run and we_looked_and_there_is_nothing. Only the last one supports a sentence such as "their pricing page did not change".

How to see a competitor's past pricing for free

The Internet Archive's Wayback Machine is the free way to see a competitor's past pricing, and its CDX API lists every capture of a page with its timestamp and a content digest. The CDX server documentation describes the parameters used here. To build a history by hand:

1. Request https://web.archive.org/cdx/search/cdx?url=competitor.com/pricing&from=2025&to=2025&filter=statuscode:200&collapse=timestamp:8, replacing the domain. The collapse=timestamp:8 parameter keeps one capture per day.
2. Ignore the digest column as a change signal. In 2025 the archive held 145 captures of Notion's pricing page and 145 different digests, one per capture.
3. Open captures at https://web.archive.org/web/ plus the timestamp and the address, starting with the first and the last of the period, and write down each plan's price.
4. When two captures disagree, open one halfway between them, and repeat until the change sits between two neighbouring captures.
5. Check the currency and the billing period on both sides before recording a change.

The archive shows only what it captured, from where it captured it. A page captured on 118 days of a year leaves gaps of several days, and a capture made from another country can show another currency.

How to get competitor changes into your own tools

Competitor changes are most useful as rows in the tools a team already uses, with one row per detected change rather than one per visit. A row that holds up has seven fields: the date detected, the page, the field that moved, the text before, the text after, the currency and period on each side, and the label (changed, currency switch, period switch, not readable). Send only the changed rows to a chat channel, and keep the rest in a log that someone reads weekly. An assistant connected over MCP can read the same rows; our plain-English guide to MCP servers explains how that connection works.

Where overads fits

overads is AI marketing automation for small teams: it drafts, reports and proposes, and by default a person approves before anything goes out. Its Competitors view has a Landing pages tab built on the rules in this guide. Adding a competitor reads its robots.txt at once and starts tracking its homepage and a guessed /pricing page, and a person can pin more pages on the same site. Both plans allow up to 15 sites per workspace and 8 pages per site.

Each visit is stored with one of twelve outcomes, including the ones where nothing was read and why. A change is recorded when the title, the headline, a button label or a price string moves, or when the body text moves past a similarity threshold, and prices are compared only when both readings look like a pricing page with amounts on them. Currency and billing-period switches get their own labels, and the tab shows the text before and after without stating a direction or a percentage. The same changes are on the API as GET /crawl/changes and on the MCP server as the list_page_changes tool, both under the analytics:read scope, and API and MCP access is included on every plan. The MCP setup guide covers connecting an assistant.

On October 18, 2026, a re-read runs when a person asks for one, at most once an hour per site. A scheduled once-a-day re-read is built and is not switched on in the live product, so a page is as fresh as its last requested reading.

What overads does not do:

• It does not run JavaScript, so prices drawn in the browser show as not readable.
• It does not sign in, fill in forms or read checkout pages, and it sends no credentials.
• It does not send an alert when a competitor page changes; changes are read in the tab, over the API or through the MCP tool.
• It does not take screenshots or compare layouts visually.
• It does not track product catalogues or marketplace prices; it watches a few pages per site.

A team whose competitors draw their prices with JavaScript, or that wants screenshot comparisons, is better served by a visual monitoring tool that runs a headless browser. A retailer tracking thousands of product prices needs a price-intelligence tool built for catalogues. A one-off look back at a competitor's pricing needs only the Wayback Machine and the steps above.