"Is this price actually a deal?" is the simplest possible question to ask about a product. It also has no simple answer. We started tracking Canadian retailer prices in 2024 because we wanted that answer for ourselves. Two years later, we have ninety days of price history on more than eight thousand baby products across Amazon.ca, Walmart.ca, and Best Buy Canada, and the question is still hard.

This post is about why. It is about the five problems that take up most of our engineering time and that explain why most price trackers you find online are either Amazon only or have data quality that nobody wants to talk about. Each section names the problem, shows a concrete example from the live dataset, and explains what Lowvyn does about it. None of these problems have clean solutions. They have trade-offs, and we try to make those trade-offs explicit so a Canadian shopper can decide whether to trust the answer they get from our tools.

1. Fake "was/now" prices

The most common pricing tactic in Canadian online retail is the inflated reference price. A car seat is listed at $549.99 with a strikethrough $799.99 next to it. The retailer says you are saving $250. The honest answer is usually closer to "you are saving five dollars and the rest is a number nobody actually paid in the last six months."

A specific example from the dataset on the day of writing. A Britax Poplar S convertible car seat on Best Buy Canada is currently $549.99. The strikethrough "was" price shown next to it is higher. Looking at our ninety day history for the same product, the price has been at or near $549.99 for most of the window, with one short period in March when it dropped to $467.49. The retailer's reference price does not appear in our records at all.

This is why a percentage discount on the product page is close to worthless as a signal. It compares the current price to a number the retailer chose, not to the price the product has actually been sold at. Our deal verdict ignores the strikethrough entirely. We compute a rolling average of the last ninety days of observations and we classify products into five buckets based on how far the current price sits below or above that average. The label "real deal" is reserved for the strictest bucket, which is a current price that sits at least twenty percent below the ninety day average. Five percent to twenty percent below average is labelled a modest deal. Within five percent of average is labelled average. Five percent to fifteen percent above average is labelled above average. More than fifteen percent above is labelled a bad time to buy. The bar for the "real deal" label is set this high on purpose, so the label means something when you see it on the index page.

The trade-off is that we sometimes miss the first day of a genuine sale. If a retailer drops the price for the first time and we have not yet observed the new price, our average still reflects the old one and the deal does not get flagged until the next day. We accept that, because the alternative is trusting the retailer's "was" price.

2. Anti bot detection on retailer sites

To compute the price history above, we need to actually read the current price from the retailer's website every day. None of the three Canadian retailers we cover publishes a public price API, so the price has to come from scraping the product page. Every retailer also actively tries to stop us from doing that.

Amazon and Walmart are the strict ones. Amazon will serve a CAPTCHA page if it sees a pattern that looks like automation, even at a low request rate. Walmart returns a soft block page that looks like the product page but has an empty price field, which is the most dangerous failure mode because a naive scraper will write a price of zero into the database. Best Buy is more permissive but still rate limits aggressively when a single IP fetches too many product pages too quickly.

The defensive work to keep daily scraping working is a treadmill. We run a residential proxy pool so the requests come from different IP addresses and look like real Canadian shoppers. We rotate user agents and respect retailer specific rate limits per second, per minute, and per day. We use a real browser engine for the harder retailers because plain HTTP requests get caught faster. When a retailer changes the DOM structure of a product page, our parser needs to be updated within hours or the daily refresh starts writing wrong prices.

Most price tracker tools you find online cover only Amazon. There is a reason for that. Amazon has been the easiest retailer to scrape at scale for years, and the affiliate program is generous enough that a thin tracker can pay for itself on the first click. Walmart and Best Buy take real engineering investment to cover at the same quality. We chose to cover all three because a price tracker that only knows about one retailer cannot tell you which retailer has the best price for the product you want.

3. Matching the same product across retailers

The Britax car seat above is listed at $549.99 on Best Buy Canada. The same physical product is listed at $424 on Walmart Canada. To tell you that, we need to know that the Best Buy listing and the Walmart listing refer to the same product. The retailers are not going to help us do that.

Each retailer assigns its own internal identifier to a product. Best Buy uses a numeric web code. Walmart uses an alphanumeric item ID. Amazon uses the ASIN. The three identifiers are entirely independent and there is no public table that maps between them. The product title is sometimes useful, but a Britax car seat appears as "Britax Poplar S Convertible Car Seat in Onyx" on one retailer and "Britax Poplar Convertible Car Seat, Stone Onyx" on another.

The cleanest signal is the UPC. Best Buy publishes UPCs reliably for most of its catalogue. Walmart publishes them on roughly seven percent of items. Amazon almost never publishes them at all. So UPC matching gets us a high confidence match for a small fraction of the catalogue, and we need a second method for everything else.

That second method is fuzzy title matching, weighted by brand and model number when those are available. We use Postgres trigram similarity on titles after normalisation (lowercasing, accent stripping, punctuation removal, pack size and colour token extraction). A pair of items from two retailers is matched if the trigram similarity is above a threshold that depends on whether brand and model number agree. We promote matches to the live cross retailer group only after a dry run has been spot checked and the precision looks acceptable.

As of this week, 9.56 percent of our active catalogue is in a multi retailer group. That number is up from less than one percent six months ago. The work to get from one in a thousand items to one in ten was nontrivial.

4. Accessories living in main product categories

The fourth problem is more subtle. A retailer puts a baby monitor in the Baby Monitors category, which is correct. The retailer also puts the universal twist mount that holds the baby monitor in the Baby Monitors category. So does the wall plate. So does the silicone case. To the retailer's catalogue these are all "Baby Monitors", but a parent shopping for a monitor sees real monitors starting around $130 and the universal mount at $12.99, and the category median ends up confused.

This matters for our deal scoring in two ways. The first is the obvious one: if you tell a shopper the median Baby Monitor price is $50, but the real monitors are $130 and the mounts are $12, the median tells nobody anything useful. The second is that one of the signals we use to flag suspicious data is whether a product is sitting at an extreme position within its own category. If the category median is $130 and an item is at $12.99, that is a strong hint that either the price is wrong or the categorisation is wrong. In practice it is almost always the categorisation.

A health scan we ran this week flagged 646 items as outliers across the catalogue. A sample of those items showed that the dominant failure mode is accessory items living in the main product category, exactly as described above. We do not deactivate these items, because the price is correct and the listing is real. We are working on a way to move accessories into subcategories without losing their price history, so the median for the main category reflects the products that shoppers actually want to compare against each other.

The honest version of "we cover twelve baby categories" is "we cover twelve baby categories, and within each of them there is a long tail of accessory listings that we are still untangling from the main product list."

5. New products with no history

The fifth problem is the one we cannot engineer our way out of. Some products are new. We have a single price observation for them. The rolling ninety day average that everything else in this post depends on is undefined for a product we have seen exactly once.

This week, 624 of the 8,100 products in our baby dataset have fewer than the minimum number of observations we need to make any claim about the price. We could pretend otherwise. Most price trackers will happily compare a single observation to itself and tell you it is "stable" or "at its all time low" or some other meaningless statement that sounds informative. We do not do that.

Instead, those products sit in a "too new to call" bucket on the research page. The user sees the current price but no deal verdict, no signal, no score. We say honestly that we have not observed the product for long enough to know whether the current price is good. Once we accumulate roughly two weeks of observations, the product graduates out of that bucket and starts getting a verdict.

This is a deliberate choice that costs us conversions. A shopper who lands on a product page for a brand new release wants an answer, and "we do not know yet" is not the answer they hoped for. We accept that cost. The alternative is to make up an answer, and a price tracker that makes up answers is a price tracker that nobody should trust the second time. The whole point of the tool is that the verdict is grounded in real observations, so when we do not have observations, the responsible thing is to say so.

What this post left out

These five problems are not a complete list. We could write another post about timezone drift in retailer scrape timestamps, currency conversion bugs on Amazon listings that switch between USD and CAD without warning, the daily comedy of catalogue duplicates where Walmart lists the same physical product under three different item IDs, and the specific failure modes of Amazon parser regressions on new page templates. The five we covered here are the ones that take up most of our engineering attention and that explain most of the gap between "a number on a product page" and "an answer to whether that number is a real deal."

If you want to see how this all lands in practice, the weekly Canadian Baby Price Index summarises the current dataset and tells you how many products are at a real deal price right now: https://lowvyn.com/research/canadian-baby-price-index. If you want to check a specific product, paste any Amazon.ca, Walmart.ca, or Best Buy Canada product URL here and we will give you the answer for that product: https://lowvyn.com/check-deal.

Both tools are free and do not require a signup. The methodology behind them is what this post is about.

The five hardest problems in Canadian price tracking