Running an ecommerce site with 3,000 products is primarily a merchandising challenge.
Running one with 3 million products is a technology problem.
At enterprise scale, many of the assumptions behind conventional ecommerce begin to break down. Product imports take too long. Search indexes struggle to stay current. Supplier feeds conflict. Pricing changes faster than caches can refresh. APIs hit rate limits. Category structures become unwieldy, and seemingly small inefficiencies suddenly have enormous consequences.
The challenge is no longer simply getting products onto a website.
It is building an architecture capable of continuously ingesting, processing, enriching, searching, serving and updating millions of records without slowing down the customer experience or creating an operational nightmare behind the scenes.
Scale changes the problem
It is tempting to think of a catalogue containing 3 million products as simply a larger version of one containing 30,000.
It isn’t.
At smaller scales, inefficient processes can remain hidden. A nightly product import that takes 20 minutes may be perfectly acceptable. A script that unnecessarily processes every record might not cause noticeable problems. A few duplicate attributes can be fixed manually.
Multiply those inefficiencies across millions of products, multiple suppliers, thousands of categories and continuous pricing and availability updates and the architecture starts to matter considerably more.
Enterprise ecommerce therefore needs to be designed around volume, velocity and change.
How many products exist matters.
But so does how frequently those products change.
Product feeds become infrastructure
For many large ecommerce businesses, the catalogue is assembled from multiple external feeds rather than maintained within a single system.
Suppliers may provide product data through APIs, XML feeds, CSV files, SFTP locations or proprietary integrations.
Each source may describe similar information differently.
One supplier might call an attribute “colour”, another “color”, another “finish”, while a fourth embeds the information within an unstructured product description.
Identifiers can differ. Categories rarely align perfectly. Images change. Products disappear. New products arrive. Stock and prices can change throughout the day.
The feed layer therefore becomes a significant piece of enterprise infrastructure.
Rather than simply importing whatever suppliers provide directly into the ecommerce platform, organisations often need an intermediate processing layer capable of validating, normalising and enriching incoming information.
That creates a controlled product model rather than allowing upstream systems to dictate the architecture of the storefront.
Full catalogue imports stop making sense
One of the first architectural assumptions that often needs reconsidering at scale is the full import.
If three million records are processed every time something changes, the system is doing enormous amounts of unnecessary work.
Enterprise systems should increasingly understand changes rather than repeatedly rebuilding the entire catalogue.
That means identifying:
- new products
- changed products
- deleted or discontinued products
- price changes
- inventory changes
- new images
- changed attributes.
Processing only the delta can dramatically reduce compute requirements and shorten the period between an upstream change and that change appearing to customers.
This becomes particularly important for businesses where stock or pricing moves rapidly.
A catalogue that is technically correct once every 24 hours may still be commercially wrong for most of the day.
APIs need architecture around them
APIs are fundamental to modern ecommerce, but connecting systems through APIs does not automatically create a scalable architecture.
At enterprise volumes, organisations need to think carefully about rate limits, latency, retries, failures and dependencies.
If every customer page request requires live responses from several external systems, one slow supplier API can potentially become a customer experience problem.
Good architecture separates processes that genuinely need real-time information from those that do not.
Product descriptions might be synchronised asynchronously.
Inventory may require near-real-time updates.
Payment authorisation genuinely needs to happen during the transaction.
Understanding those distinctions prevents an ecommerce platform from becoming unnecessarily dependent on dozens of external services for every request.
Queues, caching, event-driven architecture and background processing can all play important roles in creating resilience between systems.
Catalogue architecture becomes critical
Large catalogues quickly expose weaknesses in product modelling.
A simple product table may be sufficient for a straightforward retailer. It becomes considerably less useful when millions of products contain different combinations of attributes, variants, brands, supplier identifiers and category relationships.
A scalable catalogue needs a clear canonical product model.
The business needs to understand what constitutes a product, a variant, an offer and a supplier record.
Those distinctions matter.
The same physical product may be available from several suppliers at different prices and delivery times. Treating every supplier record as an entirely separate customer-facing product can create duplication and a poor shopping experience.
Conversely, incorrectly merging genuinely different products creates equally serious problems.
Identity resolution therefore becomes a major catalogue function.
GTINs, SKUs, manufacturer part numbers and internal identifiers can help, but real-world product data is rarely clean enough for one identifier to solve everything.
Search is not a database query
Once a catalogue reaches significant scale, product discovery needs its own architecture.
Customers expect searches to return relevant results almost instantly, regardless of whether the underlying catalogue contains 10,000 or 10 million products.
That usually means maintaining a dedicated search index rather than querying the transactional product database directly.
But indexing creates another synchronisation challenge.
When a product changes, how quickly does that change reach search?
If an item goes out of stock, should it disappear immediately?
If its price changes, how quickly should the indexed price update?
What happens when an indexing job fails halfway through?
Large ecommerce businesses need to treat search as a continuously maintained representation of the catalogue rather than a secondary feature bolted onto the storefront.
And relevance matters as much as speed.
A technically fast search that consistently puts the wrong products first is still a poor search system.
Performance problems multiply quickly
An inefficient query taking an extra 100 milliseconds may seem insignificant.
Across millions of products and large volumes of customer requests, it can become expensive.
Enterprise ecommerce performance therefore needs to be considered across several layers:
- database queries
- APIs
- application code
- search infrastructure
- image delivery
- caching
- content delivery networks
- frontend rendering.
Caching is particularly powerful, but it introduces its own challenge: invalidation.
If product information changes frequently, the system needs to know which cached objects should be refreshed without unnecessarily clearing everything.
This is another reason event-driven architectures can become valuable.
A price update can trigger the specific downstream processes required to update the catalogue, search index and relevant caches rather than relying on periodic full-system refreshes.
Synchronisation is harder than storage
Storing three million products is not particularly difficult with modern infrastructure.
Keeping three million products correct across multiple systems is much harder.
A large ecommerce operation might have versions of product information within:
- supplier systems
- an ERP
- a PIM
- an ecommerce platform
- a search engine
- a recommendation engine
- advertising feeds
- marketplaces
- analytics platforms.
The architectural question becomes: which system is authoritative for each piece of information?
Without a clear answer, systems begin overwriting one another.
The PIM changes a description. The ERP changes a price. A supplier updates availability. A merchandising team overrides a category. A marketplace requires a different product title.
Enterprise ecommerce needs explicit rules about data ownership and propagation.
Otherwise synchronisation becomes a collection of scripts nobody wants to touch.
Automation becomes essential
Manual catalogue management does not scale indefinitely.
At millions of products, businesses need systems capable of automatically identifying and resolving routine problems.
That might include:
- mapping supplier categories to an internal taxonomy
- normalising product attributes
- detecting duplicates
- identifying missing images
- flagging unusual price changes
- generating or improving descriptions
- categorising newly introduced products
- identifying feed failures
- removing discontinued inventory
- generating marketplace feeds.
AI can increasingly assist with some of these tasks, particularly where the source information is inconsistent or unstructured.
But automation should not mean removing control.
High-impact changes still need thresholds, rules, monitoring and, where appropriate, human approval.
Observability matters when millions of things can go wrong
At smaller scale, someone may notice that a feed stopped importing.
At enterprise scale, failures can be much more subtle.
Perhaps one supplier’s inventory stopped updating three hours ago.
Perhaps 40,000 products were incorrectly assigned to a category.
Perhaps a currency conversion failed.
Perhaps a search index contains yesterday’s pricing while the storefront contains today’s.
Systems therefore need observability built into them.
Teams should be able to answer questions such as:
How many products were processed today?
How many failed?
When did each supplier last update?
How long is the processing queue?
How many products are missing mandatory attributes?
How far behind is the search index?
Are API error rates increasing?
The objective is to identify problems before customers or suppliers report them.
Architecture affects commercial performance
These issues can sound deeply technical, but their consequences are commercial.
Poor catalogue architecture can mean products that cannot be found.
Slow synchronisation can mean advertising products that are no longer available.
Weak feed processing can create incorrect prices.
Poor search can push customers towards competitors.
Performance problems can reduce conversion.
Manual processes increase operating costs and make expansion into new suppliers, markets or product categories progressively harder.
Technology architecture therefore becomes part of the commercial model.
A platform capable of processing another million products without requiring another team of people has a fundamentally different cost structure from one that relies on manual intervention.
Replatforming does not automatically solve scale
When ecommerce systems begin struggling, replatforming is often considered the obvious solution.
Sometimes it is the right decision.
But moving the same architectural problems onto a newer ecommerce platform rarely fixes them.
If catalogue modelling is poor, supplier integrations are fragile and synchronisation logic is unclear, a new storefront may simply inherit those problems.
Large-scale replatforming therefore needs to consider the wider commerce architecture.
The question should not only be:
“What ecommerce platform should we use?”
It should also be:
“Where should product data live, how should it move, and which systems should be responsible for what?”
Those decisions can be more important than the storefront technology itself.
Design for the next three million products
Enterprise ecommerce architecture should not be designed solely around today’s catalogue.
Businesses need to consider what happens if they add another major supplier, expand internationally, introduce a marketplace model or double their catalogue.
If adding another 500,000 products requires rebuilding the platform, the architecture has not genuinely solved the scale problem.
The goal should be to make growth increasingly routine.
A new supplier should become another feed through an established ingestion pipeline.
A new product category should fit within a flexible catalogue model.
A new sales channel should consume existing product services rather than requiring another independent product database.
That is what scalable architecture ultimately provides: the ability to grow without technology becoming the constraint.
Enterprise ecommerce is a systems problem
At Silicon Dales, large ecommerce projects are approached as systems problems rather than simply website builds.
At significant scale, feeds, APIs, catalogue architecture, search, infrastructure, synchronisation and automation all need to work together.
Sometimes that means replatforming an existing ecommerce operation. Sometimes it means rebuilding specific components while retaining others. And sometimes the biggest improvement comes from redesigning the data architecture behind a storefront that customers barely see change.
Three million products are not inherently difficult to store.
The difficult part is keeping those products accurate, searchable, fast and commercially useful while millions of individual changes continue happening around them.
That is where enterprise ecommerce stops being a website project and becomes an engineering challenge.