Data aggregation from multiple sources into one unified system with real-time sync, cleaning and normalisation of product information. Every supplier sends data differently – different format, different units, different names for the same attribute. The goal is a single catalogue you can actually trust.

Where the real work is

Not in downloading the file, but in what follows. The same product is named differently in two feeds, units are millimetres in one and inches in another, missing values have to be filled from elsewhere, and duplicates must be matched so a better record is never overwritten by a worse one. Those rules are what makes the system valuable.

How we handle it

  • Import from XML, CSV, JSON and APIs, tolerant of changes in feed structure.
  • Normalisation of units, names and attributes into a shared data model.
  • Duplicate matching via stable identifiers and source precedence rules.
  • Change history, so it is clear when a value arrived and where it came from.
  • Quality checks – alerts on empty fields, price outliers and unexpected feed outages.

What you get

One catalogue that feeds the shop, the exports and marketing, without each channel keeping its own version of the truth. When a supplier changes something, one place updates and the change propagates everywhere. It also removes the manual spreadsheet cleaning that never quite gets done.

We use the same approach on Tyrestock.sk, where the system processes data from more than 40 tyre suppliers.

Frequently asked questions

What if a supplier sends poor-quality data?

The system does not write it into the catalogue blindly. Missing or suspicious values are flagged, filled from another source according to precedence rules, or sent for review. You also see which supplier produces the most errors – often a useful argument in negotiations.

How many sources can the system handle?

The count is not the main constraint; the variety of formats and update frequency matter more. In practice we routinely process dozens of feeds at once – on Tyrestock.sk it is more than 40 tyre suppliers with regular price and stock updates.