All companies in the video streaming sector have the same goal-- guarantee the best watching experience with efficient cost control for ingest, transcoding, and delivery. It is not an easy task and requires a lot of considerations on the Product and Engineering sides.

Measuring the Quality of Experience (QoE) is one of the methods widely used to assist this process. In this blog post, we will go through the QoE principle and functioning by relying on common use cases and metrics.

Principle

As soon as someone uses a website or an app, stakeholders need feedback in order to learn about the strengths and weaknesses of what they've built. The measurements performed for the QoE purpose are based on this same principle and are seamless for the end-user.

The QoE consists of collecting and sending users subjective data to better understand how the product is used to make the right decisions. In this case, knowing more about how videos are consumed by viewers through a first-party player.

It helps to get more insights about questions like "What will be the effects of reducing the 1080p video bitrate on the end-user engagement?" and "Is it a good approach to preload the video on the start-screen to boost its loading time?"

NOTE: QoE data must comply with local privacy policies like GDPR or CCPA. No data that breaks these laws or disrespects user privacy shall be stored without the user's explicit consent.

Functioning

The player performs periodic measures related to the video playback but also computes metrics from its own runtime context like the FCP, its loading time, and more. It then forwards all this info to a dedicated backend for later processing and analysis.

The objective is to gather enough relevant data about the "Where", the "Who" and the "How". The "Where" and the "Who" are quite important but are more "context" oriented. The "How" really refers to the QoE.

The "Where"

Locate where videos are watched to know the distribution of the audience:

• The continent → Is North America, the main continent in terms of audience?
• The country → How many videos are watched in France?

GeoIP Databases allow retrieving the continent and the country based on the user IP.

The "Who"

Identify the profile of devices used to watch the videos:

• The device_type → Are CTVs representing a significant part of the audience in Asia?
• The platform and the browser family or version → When was the latest version of macOS released? What is the part of IE11 to clean legacy code?

Even if it is not 100% trustable because it tends to be deprecated and it can be overridden, the User-Agent HTTP header exposes this info and provides, at least, good hints. There are some alternatives like UA-CH.

For a given visit, having a user_session with the number of views helps to confirm that after consecutive video playbacks, the experience remains stable and great.

The "How"

Understand the video consumption behaviours:

• The rebuffering_time → Why is the engagement so low in Asia?
• The watch_time → What is the best moment to trigger an ad-break?
• The abr_max_rendition → Is the 720p more played when increasing the default player dimensions?
• The playback_error → Why are macroblocks appearing on Safari/desktop?

There are a lot of factors that can influence video playback. Rebuffering is part of them but is the consequence of underlying problems: slow delivery, low user bandwidth, etc.

But what is rebuffering? A rebuffering describes the moment when the video playback stopped because of an insufficient amount of downloaded video bytes (buffering) without any user interactions. Seeking or switching manually to another video rendition is not considered to be rebuffering.

In other words, rebuffering is an unexpected buffering lasting N milliseconds causing the video playback to stop.

The watch_time is a duration representing the time a user has watched a video. It can be associated with the rebuffering_time to compute the rebuffering_precentage during a user_session.

ABR (Adaptive Bitrate) algorithms choose the best video renditions according to user network conditions. The abr_max_rendition describes the highest video rendition reached when ABR is enabled.

Finally, the playback_error indicates which errors have occurred during the video playback to explain sudden disruptions or degradations within the network, transmuxing, or decoding.

Conclusion

We've had a sneak peek at how QoE can be useful to drive operational decisions. Combined with the Quality of Service (QoS), this data constitutes a powerful set of valuable information to continuously ensure that video playback is optimal.

At api.video, we are committed to providing the best watching experience for all, no matter the location or the device. In another blog post, we will go through other side usages based on QoE data especially for observability to detect proactively issues and assess impact in case of outages. Stay tuned!

Resources

• https://www.akamai.com/blog/performance/enhancing-video-streaming-quality-for-exoplayer-part-1-quality-of-user-experience-metrics