ATIS - Alliance for Telecommunications Industry Solutions
Scope
"This document discusses a proposed test process for IPTV Perceptual Quality Measurements (PQMs). The Alliance for Telecommunications Industry Solutions (ATIS) IPTV Interoperability Forum (IIF) distinguishes a test process from a test plan (see [2]). The following topics relative to the proposed test process are included in this document: ? On-demand testing versus algorithm standardization. ? Content library creation. ? Publication of results. ? Operational process models. ? Operationalizing the process. ? The role of the ATIS IIF. Purpose The number of service providers rolling out IPTV services continues to increase. In order to support IPTV operations@ the need for measurements that can provide insights into the customer's perception of the quality of the IPTV content is apparent. There is a general concern that PQM solutions available today and the test process used to validate these solutions may not be adequate to support the needs of IPTV service providers. However@ some of the solutions available may be adequate for limited purposes@ such as basic reproduction quality checks (e.g.@ Peak Signal to Noise Ratio [PSNR]) or for measuring basic transmission performance@ such as traditional network Quality of Service (QoS) measurements. The general problem appears to be fourfold: 1) Different organizations create their own PQM test plans and solutions. As a consequence@ there may be cases where comparison of two models is not meaningful. 2) Lack of comprehensive requirements for a toolset for establishing@ measuring@ and monitoring Quality of Experience (QoE) for IPTV. 3) Lack of clarity in the application of measurement methods at various points in the IPTV distribution/delivery chain. 4) The current process for testing/validating methods (algorithms) is limited in scope and can be timeconsuming. The ATIS IIF addresses each of these four problems in the following documents: ? ATIS-0800025 [2]@ Test Plan for Evaluation of Quality Models for IPTV Services@ addresses the first point. This document provides a universal test plan; by standardizing the test plan@ it encourages industry developments where multiple organizations could develop QoE models@ and these models can be validated based on the same basic test plan. A basic test plan allows for ""delta"" documents to be developed for extensions that adopt the same fundamentals as this basic test plan but go into more detail for particular types of QoE models and applications. ? ATIS-0800031 [3]@ IPTV QoE Measurement Recommendations and Framework@ identifies some of the aspects in points 2 and 3 in section 5 and in its informative appendices. The document reviews a variety of IPTV QoE measurement algorithms (e.g.@ parametric and bit-stream approaches). It also provides some guidance where full-reference@ reduced-reference@ and no-reference models may be employed in an IPTV network. ? The present document@ ATIS-0800035@ Validation Process for IPTV Perceptual Quality Measurements@ addresses the fourth point. Today@ standards groups combine the test process and test plan activities with the eventual goal of a standardized PQM solution. The ATIS IIF separates these two processes. The test plan is described in [2]@ and this document describes a recommended test process. This separation provides for greater flexibility. This document describes the industry standards test process currently followed@ indicates the shortcomings@ and then suggests solutions for these weaknesses. The proposed process is flexible and is believed to address market needs better than the current process."