Compare Builder Merchants

A product does not have one price. It has a canonical identity and a set of merchant-specific listings, each carrying its own commercial offer. Treating every merchant listing as its own product produces a catalogue full of near-duplicates, and comparison stops being possible before any interface work begins.

Once identity is separated from the offer, comparison, sorting, best-price calculation, stock comparison, price history and alerts all fall out of the same model rather than being bolted onto it.

The platform has two independent paths over an owned data platform. Customer requests travel from the edge through the application and API layers. Merchant data travels its own path on its own schedule: sources, scheduled ingestion, Python workers, staging and diff. PostgreSQL is the transactional source of truth; Redis and OpenSearch are derived read infrastructure. Neither path runs through the other.

That separation is the property the rest of the architecture is arranged to protect. Everything below is a consequence of it.

Reading the paths separately: nothing on the customer path touches a merchant, and nothing on the merchant path waits for a customer request. Merchant changes reach PostgreSQL before RabbitMQ delivers isolated downstream work, including the updates that keep Redis and OpenSearch current.

Merchant acquisition runs on its own schedule, independently of customer reads. Prices are fetched ahead of time, normalized and stored locally. When a customer opens a product page, the request is served from our own application, cache, search index and database.

This is the decision the platform is built on. Aggregating live at request time makes every page as slow as the slowest merchant and as available as the least available one.

A large incoming merchant snapshot should not force every listing through downstream processing when only a subset changed. The ingestion model bulk-stages and diffs the snapshot first, so unchanged records stop at the comparison boundary.

Incoming records are bulk-loaded into a staging area rather than issuing a database round trip per product, and the comparison runs set-wise against current merchant listings. The merchant plus its own SKU is the practical stable identity within a feed when the integration contract preserves that key; sources that do not preserve it need an explicit adapter-level identity rule.

Once a merchant SKU is confidently mapped to a canonical product, that relationship stays persisted. It is only revisited when the identifying product data materially changes, which is the expensive path and the rare one.

The model expresses four separate things: what a product is, how a given merchant represents it, what that merchant is currently offering, and how that offer has moved over time. Collapsing any two of them costs either comparison or history.

The application and API layer is TypeScript. The data-processing workloads are Python and run as their own workers, so a large catalogue import never competes with the request path for the same process, and the two can be scaled against different signals.

Merchant imports are large, price updates should not block ingestion, search indexing does not need to be synchronous, and alerts should not be able to slow either. RabbitMQ sits between those workloads so each one can fail, retry and scale on its own terms.

Committing a database change and publishing a message are two operations, and the gap between them is a distributed-systems correctness boundary. If the write succeeds and the process dies before publishing, the price has changed but the cache, the search index and the alert evaluator may never hear about it.

With this pattern the event is persisted alongside the state change, so publication can be retried without losing the business event, and idempotent consumers make redelivery safe. It is set out here as the reliability boundary this class of system has to answer for, not as a published account of the production internals.

PostgreSQL stays the transactional source of truth. Product discovery runs against a read-optimized search index that is kept current asynchronously, so a browsing customer never contends with a catalogue import for the same rows.

Production traffic exceeded 100,000 requests, so customer reads were deliberately kept independent of merchant synchronization workloads. The read path and the write path scale separately and communicate through owned, durable platform state rather than live merchant calls.

The current offer answers what a customer pays now. The history answers how it got there. Keeping them separate means the read path stays a single-row lookup while the historical record grows behind it.

An external feed does not usually fail by refusing to arrive. It arrives truncated, half-parsed, or carrying a pricing column that has quietly changed meaning. A suspicious import is not allowed to overwrite production state on the assumption that it is correct.

Absence is treated with the same caution. A product missing from one feed run is not evidence that the product is gone, so records move through cautious state transitions rather than being deleted on a single observation.

The customer-facing surface is deliberately ordinary. Everything above is what makes it possible to answer these questions quickly, from our own data, while merchant catalogues keep moving underneath.