Measuring PWA Performance

Performance is one of the most important things when creating a website. Not only does it improve your SEO but more importantly, it improves your user experience. But when measuring the performance of Progressive Web Applications (PWAs), there are many different factors to take into account. On first glance, the stats you get back don’t always appear to be the best.

We’ll look in more detail at some key performance tools, what they actually measure and why they’re not always the best indicator of what your customers are actually experiencing.

The Lighthouse Score

The most famous metric right now is the Lighthouse Score. This is for two reasons: it’s one of the ~200 factors which influences the ranking of website in the Google index; and it’s very easy for anybody to view (just check out Developer Tools → Audit on a Chrome browser).

The Lighthouse Score simulates the first request of a user without any cached assets on a slow network connection (3G) with a slow device. This means that it’s optimized for static sites and totally ignores the benefits of a PWA (which has faster transitions between sites and has more interactivity). To make matters worse for a PWA, the most important metric at the moment is Time to Interactive. This means that the full client side application has to be fully loaded and initialized. For an average PWA, this is roughly 12 seconds even though in less than two seconds (usually) a user already sees a website. In fact, as soon as the server side HTML has been delivered and styled, they can use the site. But the Lighthouse Score doesn’t factor this usability into the results so a PWA will always receive a worse score a ‘traditional’ website. The score also doesn’t simulate real user behavior or their perceived performance so even though users are getting the best experience, your score doesn’t reflect it.

Most Search Engine Optimization (SEO) agencies focus on the Lighthouse Score or implement something very similar themselves and report these values when analyzing customer sites. But like all analysis, context is everything and it shouldn’t be taken at face value, especially in terms of a PWA.

The good thing about The Lighthouse Score is that the reports are excellent debugging tools. You can easily find and understand where you can optimize the performance of your site during development as you can already test it locally and on staging systems.

Lighthouse Score Metric will be Updated

The Lighthouse Score team know that it isn’t designed for PWAs and they’re in the process of releasing version 6.0. The new version includes three new metrics as well as changing the weighting importance of each metric that make up the Performance score. This is a big change and a step in the right direction but it still doesn’t take PWAs fully into account. For more info on the release and upcoming updates, click here.

Pay Attention or Ignore?

It’s a really simple and quick way to know where you need to optimize your site, just don’t take the score at face value and make sure you analyze what it really means for your customers rather than just SEO.

Tideways

Tideways (or similar tools like New Relic, App Dynamics) measures how long a backend takes to generate the response. This rendering time is also called Time To First Byte and looks at the Server Side Performance. It includes:

  1. Loading the page information
  2. Fetching all the data
  3. Server Side Rendering (only on the first response)

Monitoring in Tideways doesn’t take into account how long it takes to deliver the data to the user (especially important on slower connections) or how long the browser takes to render the data. It also doesn’t measure the transfer of static assets (like CSS, JavaScript and Images) because they basically don’t consume any time on the server (sub-millisecond).

Tideways is another great tool to debug performance issues, especially with data integrations like APIs, and it’s a great tool for Developers. The server side performance also influences all the other metrics – from Performance to SEO – since basically everything that happens in the client depends on data from the server. The problem though is that the Time To First Byte metric doesn’t have a major influence on the other metrics.

Lighthouse also measures Time to First Byte but they include the data transfer size which isn’t measured by Tideways. So in some cases where raw data (stream data, page metadata, …) is transferred next to the HTML, slow connections will affect this score and it can vary a lot between Tideways and Lighthouse.

Pay Attention or Ignore?

It’s great tool for Developers and to improve performance but it won’t give you the full picture.

Chrome User Experience Report

The Chrome User Experience Report (CrUX) provides by far the most holistic metric and also is part of the search engine ranking score. It’s often neglected (especially compared to the Lighthouse Score) because it isn’t so easily visible. The performance data from real users using the Chrome browser is aggregated for live websites and this data is analyzed and available for download here.

So Which One Should I Use?

As with most things, there’s no perfect way to measure performance when it comes to PWAs. The most important thing is to understand how your website works, and make sure that your business does too, that way you can better understand the performance scores your site is getting.

Use your site yourself so you experience exactly what your users experience. Use different devices on different networks so you can see first hand where you need to make improvements. Then when you know what you want to measure, find the best tool to measure exactly that. It’s better to use a set of tools to get the full picture and optimize your site, rather than just relying on one measurement.

PWAs in General

After all this, wouldn’t it be easier to just avoid using a PWA? If your only goal is to have a good Lighthouse Score, then yes. But if your business KPIs include having the best user experience across all devices, increasing your sales, creating brand advocacy, etc. having a PWA does all these things and much more.

With Frontastic, a PWA comes as standard so you can develop for Mobile devices first and create amazing digital moments for your users. We want to help you to optimize your PWA and like we said earlier, the best way to do this is to understand what’s happening when a user visits your site.

When a user makes a first ever request to a Frontastic website, they’ll receive the HTML of the website (done by our Server Side Rendering). Besides this, they need a few other things to be able to use the website:

  1. The styling information (CSS)
  2. The data to initialize the PWA
  3. The JavaScript which is basically the PWA
  4. The images (videos, etc.) which are used on the website

After that, we’ll only load the data for every subsequent page (not of the assets any more, except for chunked Tastic or images used on those pages) if the user clicks on a link inside the PWA.

We also use a ‘Service Worker’. This ensures even all the styling information (CSS), JavaScript and static images are cached inside the browser. This means that the data is already there for returning visitors and users will get an even better performance. We also use technologies like HTTP/2 push to optimize the transfer of data from the server to the client.

For more information on how you can improve your performance as well as optimizing your SEO with Frontastic, see our help articles at https://docs.frontastic.cloud/.

It’s the most realistic report for providing a sensible overview on site performance including Time to First Byte, First Paint, Cumulative Layout Shift and much more. However, it’s not easily digestible, and it needs to be processed and visualized. The other issue is that it only collects data for live sites and there’s no way of evaluating your site before launch.

Pay Attention or Ignore?

If you can get this data transferred into an easy to read report, it’s well worth it and should be the go to report for any live site.

Henning Emmrich

Henning is a Software Expert with strong enterprise skills. He’s worked in many management positions like Product Management, Marketing, Business Development and Digitalization. He’s one of the Co-Founders of Frontastic and works as Integrator and COO.

Stay In The Loop

Subscribe to our newsletter to keep up to date on all the latest Frontastic news.

Related