2011/1 Vol. J94 B No. 1 [5] PESQ (Perceptual Evaluation of Speech Quality) [6] Wideband-PESQ [7] PEAQ (Perceptual Evaluation of Audio Quality) [8] SDT

Size: px

Start display at page:

Download "2011/1 Vol. J94 B No. 1 [5] PESQ (Perceptual Evaluation of Speech Quality) [6] Wideband-PESQ [7] PEAQ (Perceptual Evaluation of Audio Quality) [8] SDT"

しょうぶままだ
5 years ago
Views:

1 a) Multimedia Quality Estimation Model for Video Streaming Services Toshiko TOMINAGA a), Kazuhisa YAMAGISHI, Takanori HAYASHI, and Akira TAKAHASHI IP (QoE Quality of Experience) HDTV QoE HDTV 1. IP (QoE Quality of Experience) [1] IP QoE NTT Service Integration Laboratories, NTT Corporation, Midori-cho, Musashino-shi, Japan a) toshiko.tominaga@lab.ntt.co.jp [2], [3] 1 / IP QoE E-model [4] E-model R E-model B Vol. J94 B No. 1 pp c

2 2011/1 Vol. J94 B No. 1 [5] PESQ (Perceptual Evaluation of Speech Quality) [6] Wideband-PESQ [7] PEAQ (Perceptual Evaluation of Audio Quality) [8] SDTV ITU-T J.144 [9] PC QCIF VGA IP ITU-T J.246 [10] J.247 [11] [12] [16] ITU-T J.148 [17] Hands [18] [19] [18], [20], [21] [22] [23], [24] ITU-T G.1070 [25] HDTV (1) / (2) (3) ITU-T J J.148 Aq Vq 50

3 1 J.148 Fig. 1 J.148 Multimedia quality estimation model. 2 Fig. 2 Multimedia quality estimation model. J.148 Aq (Vq) Vq (Aq) J E-model PEAQ J.246 J ITU-T P.920 [26] Appendix 1 Aq (Vq) Vq (Aq) Aq Vq 3. 1 A ITU-T P.920 Appendix B 2 A 51

4 2011/1 Vol. J94 B No. 1 Table 1 1 Subjective assessment test environment. Video monitor 32-inch CRT monitor SONY BVM-D32E1WJ Viewing distance 3H (H picture height) about 110 cm Illumination intensity About 30 lx Listening equipment Stereo loud speakers ONKYO GX-77M Ambient noise Hoth noise at 35 db (A) at receiving side 4 Table 4 Quality parameter conditions. Video Video bitrate 14, 10, 6 Mbit/s Quality Audio bitrate 192 kbit/s Random packet 14 Mbit/s: 0, 0.010, 0.030, loss ratio (%) 10 Mbit/s: 0, 0.015, 0.040, Mbit/s: 0, 0.020, 0.080, Audio Video bitrate 14 Mbit/s Quality Audio bitrate 192 kbit/s Random packet 0, 0.100, 0.200, loss ratio (%) 2 Table 2 Codec conditions. Video codec H.264 High profile level 4.0 Group of picture (GOP) M = 3, N = 15 Frame rate 30 fps Video resolution Audio codec MPEG-2 audio AAC-LC 3 Table 3 Contents conditions. Video Audio 1 European market (No.7 [27]) BGM (No.7 [27]) 2 Harbor scene (No.12 [27]) BGM (No.30 [27]) 3 Crowded crosswalk (No.30 [27]) Pop [28] 4 Green Leaves (No.23 [27]) Piano [28] 5 Weather report (No.39 [27]) Instrumental [28] 6 Street car (No.10 [27]) BGM (No.10 [27]) 7 Woman s speech Woman s voice 8 Woman with toy block Woman s voice B MPEG-2 AAC H CRT 1 2 ITU-R BT.1210 [27] BGM (SQAM) [28] 3 / 188 Byte MPEG-2 TS (Transport Stream) IP MPEG-2 TS IP 1356 Byte 4 10 [29], [30] kbit/s 14 Mbit/s IP 70 TS TS IP 14 Mbit/s 10 IP IP / / 4 VTR 52

5 5 5 Table 5 Five-grade impairment scale DSIS Double Stimulus Impairment Scale Method [31] 5 5 (DMOS Degradation Mean Opinion Score) HDTV B 3. 2 DMOS A DMOS V DMOS MM 3 A DMOS B DMOS 3(a) A DMOS B DMOS 5% A B DMOS Fig. 3 DMOS relationships between methods A and B (b) 5% 3(c) HDTV 53

6 2011/1 Vol. J94 B No. 1 HDTV 4. HDTV DMOS A DMOS V Fig. 4 Distribution of DMOS Aand DMOS V. 6 Table 6 Multiple regression analysis results for parameters. RMSE (r) DMOS A NA DMOS V NA DMOS ADMOS V NA DMOS A, DMOS V NG DMOS A, DMOS ADMOS V NG DMOS V, DMOS ADMOS V OK DMOS A, DMOS V, NG DMOS ADMOS V (1) DMOS MM = αdmos A + βdmos V + γdmos ADMOS V + δ (1) 1 DMOS MM 5 DMOS ADMOS V α δ 6 7 (RMSE) 6 5% NA 54

7 5 Fig. 5 Accuracy of multimedia quality estimation model. NG OK RMSE DMOS V DMOS ADMOS V (2) DMOS MM =0.155DMOS V DMOS ADMOS V (2) (2) 4. 3 [19] QoE 6 Fig. 6 Boundary of each category by psychological scale. DMOS 6 DMOS DMOS = 1 DMOS = 5 7 DMOS A DMOS V DMOS MM PS A PS V PS MM 6 7 (3) PS MM =0.196PS A PS V PS APS V (3)

8 2011/1 Vol. J94 B No. 1 8 Fig. 8 Accuracy of multimedia quality estimation for psychological scale data Fig. 7 Psychological scale versus DMOS. 7 Table 7 Multiple regression analysis results using psychological scale. RMSE (r) PS A NA PS V NA PS APS V NA PS A, PS V OK PS A, PS APS V NG PS V, PS APS V OK PS A, PS V, PS APS V OK MPEG-2 TS IP TS TS IP IP HDTV IP

9 1 (SENNHEISER HD25-1) Table 8 Quality parameter conditions for model qualification. Video Codec H.264 Resolution Bitrate 14, 10, 6 Mbit/s Frame rate 30 fps Audio Codec MPEG-4 AAC-LC Bitrate 192 kbit/s Random packet loss ratio (%) 14 Mbit/s: 0, 0.015, 0.035, Mbit/s: 0, 0.020, 0.050, Mbit/s: 0, 0.030, 0.070, DMOS A DMOS V 9 IP 10 Fig. 10 Accuracy of multimedia quality estimation for system dependency. 9 DMOS A DMOS V Fig. 9 Distribution of DMOS A and DMOS V for qualification condition. 11 Fig. 11 Contents for verification. 57

10 2011/1 Vol. J94 B No. 1 9 Table 9 Contents detail. (a) BGM (b) BGM (c) CG BGM (d) BGM (e) 4 (f) BGM (g) CM BGM (h) (2) DMOS A DMOS V (2) Fig. 12 Accuracy of multimedia quality estimation for unknown contents. 6. HDTV HDTV 58

11 (SCOPE) ( ) [1] ITU-T Rec. P.10, Vocabulary for performance and quality of service, July [2] A. Takahashi, D. Hands, and V. Barriac, Standardization activities in ITU on QoE assessment of IPTV, IEEE Commun. Mag., pp.78 84, Feb [3] QoE A vol.j91-a, no.6, pp , June [4] ITU-T Rec. G.107, The E-model: A computational model for use in transmission planning, April [5] A. Takahashi, A. Kurashima, and H. Yoshino, Objective assessment methodology for estimating conversational quality in VoIP, IEEE Trans. ASLP, vol.14, no.6, pp , Nov [6] ITU-T Rec. P.862, Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs, Feb [7] ITU-T Rec. P.862.2, Wideband extension to Recommendation P.862 for the assessment of wideband telephone networks and speech codecs, Nov [8] ITU-R Rec. BS.1387, Method for objective measurements of perceived audio quality, Nov [9] ITU-T Rec. J.144, Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference, March [10] ITU-T Rec. J.246, Perceptual visual quality measurement techniques for multimedia services over digital cable television networks in the presence of a reduced bandwidth reference, Aug [11] ITU-T Rec. J.247, Objective perceptual multimedia video quality measurement in the presence of a full reference, Aug [12] B vol.j88-b, no.4, pp , April [13] H. Pinson and S. Wolf, A new standardized method for objectively measuring video quality, IEEE Trans. Broadcast., vol.50, no.3, pp , Sept [14] H.R. Sheikh, M.F. Sabir, and A.C. Bovik, A statistical evaluation of recent full reference image quality assessment algorithms, IEEE Trans. Image Process., vol.15, no.11, pp , Nov [15] B vol.j90-b, no.10, pp , Oct [16] P.L. Callet, C. Viard-Gaudin, and D. Barba, A convolutional neural network approach for objective video quality assessment, IEEE Trans. Neural Netw., vol.17, no.5, pp , Sept [17] ITU-T Rec. J.148, Requirements for an objective perceptual multimedia quality model, May [18] D.S. Hands, A basic multimedia quality model, IEEE Trans. Multimed., vol.6, no.6, pp , Dec [19] S. Tasaka and Y. Ito, Psychometric analysis of the mutually compensatory property of multimedia QoS, Conf. Rec. IEEE ICC2003, pp , May [20] A. Rimell and A. Owen, The effect of focused attention on audio-visual quality perception with applications in multi-modal codec design, IEEE ICASSP 2000, vol.4, pp , June [21] S. Winkler and C. Faller, Perceived audiovisual quality of low-bitrate multimedia content, IEEE Trans. Multimed., vol.8, no.5, pp , Oct [22] M.R. Frater, J.F. Arnold, and A. Vahedian, Impact of audio on subjective assessment of video quality in videoconferencing applications, IEEE Trans. Circuits Syst. Video Technol., vol.11, no.9, pp , Sept [23] 59

2011/1 Vol. J94 B No. 1 CQ2008-33, Sept. 2008. [24] M.N. Garcia and A.

1070, Opinion model for videotelephony applications, March 2007. [26] ITU-T Rec. P.

1210, Test materials to be used in subjective assessment, Oct. 1995.

Hayashi, Non-intrusive packetlayer model for monitoring video quality of IPTV services, IEICE Trans. Fundamentals, vol.e92-a, no.12, pp.3297 3306, Dec.

12 2011/1 Vol. J94 B No. 1 CQ , Sept [24] M.N. Garcia and A. Raake, Impairment-factorbased audio-visual quality model for IPTV, IEEE QoMEX 2009, pp.1 6, July [25] ITU-T Rec. G.1070, Opinion model for videotelephony applications, March [26] ITU-T Rec. P.920, Interactive test methods for audiovisual communications, May [27] ITU-R Rec. BT.1210, Test materials to be used in subjective assessment, Oct [28] European Broadcasting Union, SQAM (sound quality assessment material) recording for subjective tests, April [29] K. Yamagishi and T. Hayashi, Non-intrusive packetlayer model for monitoring video quality of IPTV services, IEICE Trans. Fundamentals, vol.e92-a, no.12, pp , Dec [30] N. Egi, T. Hayashi, and A. Takahashi, Parametric packet-layer model for evaluation audio quality in multimedia streaming services, IEICE Trans. Commun., vol.e93-b, no.6, pp , June [31] ITU-T Rec. P.911, Subjective audiovisual quality assessment methods for multimedia applications, Dec NTT ITU-T SG12 14 TTC 17 ITU-T SG12 13 H21 ITU-T SG12 16 (TTC) IP 17 ITU NTT IPTV NTT

10_08.dvi

10_08.dvi 476 67 10 2011 pp. 476 481 * 43.72.+q 1. MOS Mean Opinion Score ITU-T P.835 [1] [2] [3] Subjective and objective quality evaluation of noisereduced speech. Takeshi Yamada, Shoji Makino and Nobuhiko Kitawaki