August 2, 2021
December 31, 2021
Share this page:
The premise of this project is that by collaboratively combining logs, metrics, and distributed request traces throughout the set of services necessary to support video playback we can dramatically increase observability of the overall streaming video experience and drive increased quality of service for content providers and their service providers to mutual benefit.
Content providers, who frequently rely on third party software and services (Players, CDNs, Origin Services), struggle to achieve the observability necessary to achieve their QoE goals. In short, they are responsible for the entire customer experience but only able to fully control or even observe what they themselves instrument and what their service providers are able/willing to share. In addition, service providers struggle to provide optimal experiences to their customers (content providers) due to the same observability challenge. In short, telemetry is fragmented and siloed making it virtually impossible to get the complete architectural or operational picture. Data sharing and correlation between content providers and CDN service providers are emerging trends in the streaming video industry. Several CDNs offer logging data feeds with varying levels of sophistication. Content providers are starting to look at CDN and play data together to better understand QoE. Conventions like the Common Media Client Data (CTA-5004) specification have established methods for how player session info can be relayed to CDNs as metadata via HTTP requests and correlated with cache sessions logs. This is both a huge step forward and yet insufficient for achieving sustained, high levels of QoE. Shared, correlated logs and metrics are not enough to fully understand where things fail or why. Logs provide fine grained, event-level, service-specific telemetry to perform deep, localized, analysis. Metrics, which may be derived from logs or independently generated, are summarized calculations which may act as signals for quality issues but they frequently don’t help operators and engineers develop a deep understanding of the services they support or rapidly address production issues. For that we must add distributed request tracing, the third leg of the observability stool. Distributed Request Tracing builds upon logs and metrics, surpassing their utility by creating an observational map of a distributed system, frequently a cloud-based, microservice architecture. The emerging standard for distributed request tracing is the OpenTracing project (recently merged with OpenSensus into the new OpenTelemetry project).
There is not a document currently associated with this project.
Goals and Objectives
The deliverable for the first phase of this initiative is a detailed internal report and presentation to the SVA. This report will include specifics on the following:
- The instrumentation and implementation performed to enable observability through logs, metrics, and traces across the participating services.
- The scenarios simulated within the instrumented set of services.
- The tools, visualizations, and analytics developed to enhance observability
- The results of the simulations and how the instrumentation and tools developed enable rapid resolution of real-world problems.
- Optionally (depending on the results) a white paper and presentation may be developed for public consumption.
- Optionally, build on the work done in phase 1 to add more complex use cases like live streaming, ad delivery, or other scenarios that might be triaged out of phase 1.
The following members have contributed to this project. Click on their name to visit their profile. If the have not published their profile, the link will redirect to their LinkedIn profile.