How to optimize data aggregation

Performance is key for all sites. Not only does it improve SEO, but it increases sales and customer happiness. But companies must recognize the challenges that are involved. In this series, we’ll go through the main areas of site performance, how you can improve the performance of your site, and the trade-offs you’ll have to make when it comes to performance versus customer experience.

In this article of the series, we’ll look at optimizing data aggregation.

Why do I need to aggregate my data?

The heavy lifting of collecting and preparing all data required for the frontend happens in the backend. Given the amount of data it takes to power an eCommerce site, especially the amount of data from different APIs, the backend needs to be super quick to deliver this to your users to ensure they don’t have a slow user experience.

How do I aggregate my data?

Aggregating your data is a hugely underestimated task, and it’s quite a complex part of every modern frontend stack. It’s even more complex if your business model is based on multiple data sources and not just to display data from a single endpoint. For example, marrying content from a CMS with product data.

Frontastic provides the middle layer to aggregate all the data you want to see in your frontend, based on the API configuration done by storefront managers. Since we also let storefront managers attach arbitrary data to a single page, use complex queries on the backend systems, and use data from any number of systems, optimizing and monitoring the fetching of this data is a central aspect of Frontastic.

How does data aggregation work with Frontastic?

The 1st optimization we apply is only fetching the data streams which are actually used by the Tastics (frontend components) on a page.

The 2nd, and maybe most important, optimization is that we fetch all data streams in parallel. One of the key benefits of Frontastic is that it enables you to start with multiple product data streams and then combine it with content from a CMS and any additional custom data (for example, recommendations or products), basically whatever powers your business domain. This can be data from many different systems. While we favor fast API first systems, you still don’t want to wait for the sequential loading of all this data.

The 3rd optimization is that we automatically limit the number of retrieved items if the Tastics (frontend components) include settings for this. If multiple Tastics use a single product data stream but each of them only displays 3 products, we’ll only fetch 3 products from the API.

But the performance still depends on the API performance. It will only be as fast as the slowest API response, plus very few milliseconds for the internal processing of the data. On top of this, Frontastic handles errors in those APIs gracefully.

What else should I watch out for when handling data aggregation?

Frontastic also allows you to configure caching for each of these APIs. This can be beneficial, but caching can also be dangerous. Especially if you have products that can quickly run out of stock at any time, and you don’t want any listings to include these products. In such a case, we expose a granular caching configuration to storefront managers because we believe they know best about how quickly specific products might sell out.

Besides all the out-of-the-box optimizations Frontastic already implements, we also give you the tools to further optimize the data stream handling, but this will be the topic of another post.