Delay controlled wireless video playout system

www.ietdl.org Published in IET Communications Received on 23rd July 2009 Revised on 13th November 2009 doi: 10.1049/iet-com.2009.0467

In Special Issue on Video Communications over Wireless Networks ISSN 1751-8628

Delay controlled wireless video playout system C. Rui1,2 M. Zhengkun1 J. Liangbao2 H.F. Rashvand3 1

Jiangsu Provincial Key Laboratory of Wireless Communication, Nanjing University of Post and Telecommunications, Nanjing 210003, People’s Republic of China 2 School of Communications Engineering, Nanjing Institute of Technology, Nanjing 211167, People’s Republic of China 3 School of Engineering, University of Warwick, Coventry CV4 7AL, UK E-mail: [email protected]

Abstract: Best-effort streamed video playout representing the most popular and cost-effective multimedia application today is unfortunately very sensitive to any fluctuations in packet loss and delay. There are always further problems for the provision of real-time playout applications over wireless links because of the complicated and highly variable nature of the channel and challenging quality issues for adopting the service in global scales, which necessitates adopting a new and effective buffer mechanism. As a solution to this problem, to smooth out the playout for quality viewing and improved overall systems’ performance, in this study the authors propose a new adaptive media playout (AMP) system that makes the most effective use of a packet-delay prediction algorithm. Here, as the prediction method makes decisions based on the delay interdependency of the adjacent packets the differential auto-regression delay prediction model is used for working out the estimation. The proposed algorithm also adopts more refinement changing steps in the adjustment process and takes into accounts any packet losses with deadline constraint. The simulation results show that the proposed method outperforms the AMP-live method in reducing the average buffer delays as well as enhancing package loss rates for both overflow and underflow.

1

Introduction

Video streaming over interference-prone and erratic wireless channels is continuously growing. Owing to the relatively rapid variation of the physical channel within the wireless environment, the packet loss probability at the receiving side arises accordingly. Buffering is a common approach to reducing a system’s sensitivity to short-term fluctuations of data arrival rate by absorbing variations in end-to-end delay. However, buffering will inherently lead to latency especially with large buffer size. Latency as pre-roll delay, that is the time it takes for the buffer to fill with data and start the playout, is tolerable in practice. But for video streaming of live events or interactive communications, the latency has remarkable impact on the user’s experience of session quality. Furthermore, buffering is generally useful under the condition only when the client keeps the mean data arrival rate at not less than the playout rate. If rate offered by the channel falls below the playout rate, a buffer will underflow [1] and users will suffer from discontinuous playback. On the other hand, if playout is adjusted to too 1348 & The Institution of Engineering and Technology 2010

slow, a buffer overflow will occur leading to data loss and hence the visual quality degradation. Prior works have been done to resolve these problems. Researchers have proposed different solutions, which can be grouped into two major classes in decoder playout strategy. One class concentrates on the initial state of the decoder with the emphasis placed on buffering as less data as possible so as to introduce less playout delay without sacrificing the quality of service (QoS) [2, 3]. The solution of Stockhammer et al. minimises the initial playout delay and the decoder buffer size without consideration of playout speed adaptive adjustment. The result is that the latency is reduced at the cost of the increasing probability of decoder buffer overflow/underflow. The other class concentrates on the adaptive adjustment of playout. The solutions employ buffer fullness-aware adaptive media playout (AMP) speed adjustment or content-aware AMP algorithms [4 – 7]. Chuang et al. [6] set a buffer threshold to indicate the buffer fullness. The IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

www.ietdl.org threshold is dynamically determined according to the predicted channel quality, higher threshold for poor network condition and lower threshold for a good one. Yang et al. [7] propose AMP mechanism based on the channel quality to have an early adjustment of the playout rate. Unfortunately, the effect of these adaptations is not satisfying because of the difficulty in reliable prediction of the channel quality, especially for wireless communication networks. Unlike the above buffer-level-based approach, Li et al. [8] proposed a scheme of content-aware AMP integrating the information from MAC layer and video application layer. However, the high computational complexity of obtaining the scene characteristics makes it problematic for real-time applications. In this paper, a novel scheme based on differential autoregressive (DIAR) delay prediction algorithm is proposed. In this scheme, the playout speed is adjusted according to the buffer state and the delay prediction of the approaching packets. By removing the correlation in delay sequences, the delay prediction is reliable and the probability of the buffer overflow/underflow is significantly decreased. And the notable low complexity and low computation cost of the scheme makes it suitable for energy-limited mobile terminals and real-time applications. The remaining part of the paper is organised as follows. Section 2 reviews the conventional AMP control schemes. Section 3 describes the technical and the implementation details of the proposed new-AMP algorithm based on delay prediction. The simulation results are discussed and the performance comparisons are presented in Section 4. Finally, some concluding remarks are given in Section 5.

2

Conventional adaptive playout

AMP is a receiver-based buffer control technique that adjusts the playback frame rate to minimise the probability of buffer outage. Typically, the AMP-based buffer control is performed in two steps. The first step is to determine the threshold for control activation. It chooses a suitable threshold of buffer fullness to prevent buffer outage. The second step is to compute the playout rate based on the relationship between current buffer fullness and the predetermined threshold. Fig. 1 shows non-adaptive media playout (non-AMP) scheme, which begins to play the media content as soon as the number of packets in playout buffer reaches a certain value Nmax and later on plays at a constant speed. The playout parameters Nmax and the interval value are determined based on the average packet transferring speed dave,transfer . They are set when the connection is established and will not change during the playout. The IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

Figure 1 Non-AMP playout scheme

AMP allows the client to buffer less data, thus introducing less delay to achieve given playout reliability. The receiver should slow down its media playout speed when the ‘new’ packets do not arrive from the transmitter in time (in other words, the number of packets in playout buffer is running down because of the slow arrival rate). Otherwise, the playout will pause for a while to accumulate sufficient packets in the buffer to ensure the smooth playout later. On the other hand, when the packets arrival rate is faster, reaching the other extreme so that an increasing number of packets have been pushed into the buffer, the playout speed should be increased in order to make best use of the highquality channel. AMP is widely used in media playout, which has three modes as follows [9]: 1. AMP-initial: The client begins playing media before the buffer is filled to the usual target level, and allows the buffer to fill to the target level over time by initially playing the media slower than normal. The buffer fills over time because the data consumption rate during slowed playout is smaller than the arrival rate of the media packets, assuming that during normal playout the source rate and the channel throughput match the data consumption rate at the decoder. Once the target level is reached, the playout speed returns to normal. This technique allows fast switching between different programs or channels without sacrificing protection against adverse channel conditions after the initial buffer is built up. 2. AMP-robust: AMP-robust increases the robustness of the playout process with respect to adverse channel conditions. In this mode, whenever the playout buffer backlog dips below a threshold level, the playout rate is 1349

& The Institution of Engineering and Technology 2010

www.ietdl.org reduced. When the backlog regains the threshold level, playout resumes normally. With AMP-robust, if the mean data arrival rate at the client remains the same as the source rate but fluctuates a little around the mean, slow playout periods increase the mean backlog of the client leading to more robust protection against underflow. This is desirable for stored programs, where latency is not noticeable after the pre-roll buffering period. 3. AMP-live: The client slows down playout during bad channel periods but may also play media faster than the normal during good channel periods to eliminate any viewing latency that has accumulated during periods of slowed playout. By playing the media faster or slower than the normal, the mean viewing latency can be reduced for a given probability of buffer underflow. For an AMP-live, whenever the buffer occupancy falls below the target level, the playout is slowed down. When the occupancy is greater than the target level, media is played faster than the normal to eliminate excessive latency. The third mode, AMP-live is usually used in real-time playout, where there is high requirement of as little delay as possible because AMP-live is known as very efficient in reducing the playout delay. AMP-live algorithm is illustrated in Fig. 2. The amount of packets in the playout buffer is used as an indicator. The playout speed is increased by 25% if the number of inbuffer packets exceeds the pre-defined value Nmax , which means that the calculated playout time of the ith packet tr,i will be smaller than the scheduled playout time tp,i which is based on the pre-determined frame interval. Contrarily, when the number of packets stored in the buffer is less than Nmax , the playout speed is reduced by 25%. The rational of choosing 25% as an adjustment step lies in the

Figure 2 AMP-live playout scheme 1350 & The Institution of Engineering and Technology 2010

fact that informal subjective tests have shown that reduction or increase of playback rate up to 25% is unnoticeable to the user [10].

3

AMP based on delay prediction

3.1 Proposed methodology AMP-live model can reduce the buffer delay with low packet loss rate (packet losses because of inability to arrive before the deadline) and low underflow rate. The method, however, has its disadvantages: 1. Only three steps of speed are defined for the playout adjustment: normal playout speed, 25% faster and 25% slower than normal. They are difficult to adapt to the current network packet arrival rate. 2. The adjustment of playout speed only considers the current status of channel and buffer, without considering the trend of the status. 3. The scheme does not take into account the requirements of real-time video applications in setting the playout time. Usually, the end-to-end delay is not allowed to be over 400 ms. This restriction is not considered in AMP-live. The improved new-AMP algorithm with better performance is shown in Fig. 3. Compared with the existing AMP-live method, the proposed algorithm gets rid of the above three drawbacks in the following way: 1. Refine the playout speed adjustment step: A refined adjustment step in playout speed is employed to replace the simple adjustment scheme in AMP-live. For example, if the adjustment step is set as 5 ms (i.e. 12.5%), then the media playout speed can be adjusted by increasing or decreasing 5 ms when required. There is an upper/under limit for the playout speed corresponding to the slowest and fastest playout speeds, respectively. No further adjustment will be done once the playout speed reaches its limits. 2. Evaluate the future buffer status by predicting the packet delay: The proposed algorithm adjusts the playout speed by predicting the future buffer status, rather than the current buffer status. As shown in Fig. 3, once the current packet (packet i) is reached, the algorithm predicts the delay of packets i + 1, i + 2, . . ., i + j based on the previous delay values. According to the operating playout interval value, the expected playout times of packet i to packet i + j denoted as tp,i , tp,i+1 , . . ., tp,i+j are calculated. Then the arriving time of packet i + j denoted as tr,i+j is calculated by the prediction of its delay and compared with the playout time of those packets stored in the buffer. Based on the above calculations, the number of packets in the buffer at time tr,i+j is counted. If the number of buffering packets IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

www.ietdl.org

Figure 3 New-AMP playout scheme

is greater than Nmax , the playout speed is increased by decreasing the playout interval value and vice versa. 3. Take the packet deadline restraint into account when setting the packet playout time: Generally, the maximum end-toend delay tolerance is specified by users while establishing the application connection. According to the operating playout interval, the playout time of each packet is scheduled. Firstly, the playout time t1p,i is calculated. Then t2p,i is calculated by means of the maximum delay tolerance BDmax (which is calculated by the mean delay of those previous packets) plus the packet arrival time tr,i (i.e. t2p,i ¼ tr,i + BDmax). The playout time of packet i is then computed by tp,i = min(t1p,i , t2p,i ).

3.2 Auto-regressive moving average (ARMA) methodology The critical step of the proposed new-AMP scheme is the prediction of the playout delay, tNmin +1 , tNmin +2 , tNmin +3 , tNmin +4 , as indicated in Fig. 3. Differential delay prediction IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

algorithms have been applied in multimedia communication. There are two main algorithms: linear prediction and statistical analysis prediction. The former is attractive because of its low complexity. Auto-regressive (AR) and normalise least mean square (NLMS) are two widely used linear prediction methods. AR method employs ARMA model, which has been adopted by internet engineering task force (IETF). In AR model, the input sequence should be stationary random process to ensure the prediction accuracy. NLMS is based on least minimum square theory and is able to adjust the buffer delay according to the network delay and jitter. So NLMS is more accurate than AR algorithm with higher computational complexity. As is known, the multimedia packets have to be queued when passing through a series of routers, which will lead to correlation in delay sequence. For the wireless channel, the effect of physical channel impairment will also be reflected in the interdependency of packet delay sequence to some extent but with short-term fluctuation. As such the packet 1351

& The Institution of Engineering and Technology 2010

www.ietdl.org delay sequence is in fact non-stationary time series, violating the assumption of ARMA model and thus leading to low accuracy. To remove the correlation in delay sequences, a DIAR algorithm is proposed by the authors to effectively deal with the non-stationary random time series [11]. Integrating DIAR with traditional AR model is referred to as auto-regression integrated with moving average (ARIMA) model. General d-order ARIMA process is calculated by superimposing the stationary series d times. It uses the differential ∇zt = zt − zt−1 = (1 − B)zt [the inverse operation of S = ∇−1 = (1 − B)−1 ] to calculate the input time series in order to approach the stationary condition. Then ARMA prediction model can be applied. Improved accuracy of prediction can be reached. The detailed method is as follows. Firstly, using the packet delay series as the input, that is, performing differential operation to AR algorithm, we have di − di−1 = a(di−1 − di−2 ) + (1 − a)(ni − ni−1 )

(1)

Denoting Ddi = di − di−1 , and Dni = ni − ni−1 , (1) can be re-written as Ddi = aDdi−1 + (1 − a)Dni

(2)

Then the delay differential oscillation and end-to-end delay can be calculated by  Dvi = aDvi−1 + (1 − a)|Ddi − Dni | Di = Di−1 + DDi = Di−1 + Ddi + bDvi = ni−1 + Ddi + bDvi

Figure 4 PSER of AR, NLMS and DIAR methods

Columbia, Harvard, MIT, Stanford, USC and Washington. Fig. 4 shows the prediction signal to error ratio (PSER) values from simulations based on predictive methods: AR, NLMS and DIAR with detailed data in Table 1.

4.2 Testing of ARIMA methodology In order to compare the prediction accuracy, PSER is used as the performance metrics, which is similar to peak signal to noise ratio  PSER = 10 log10

(3)

 E[y2 (n + 1)] (dB) E[e2 (n + 1)]

(4)

Formulae (2) and (3) are the basis of AR algorithm, where di is the auto-regression predicted value, ni is the delay of packet i, a is the model parameter, vi is the network delay jitter and Di is the end-to-end delay.

where y(n + 1) is the delay of packet n + 1, yˆ (n + 1) is the predicted delay of packet n + 1 and e(n + 1) = y(n + 1) − yˆ (n + 1) is the prediction error.

4

It can be seen from Fig. 4 that the methods of NLMS and DIAR outperform the method of AR in all simulations with eight groups of testing data. The prediction precision of DIAR is about 5 – 10 dB higher than that of AR algorithm without increasing the computational complexity.

Simulation results

4.1 System under study The testing data used in the simulation, totalling to 4 000 000 messages, is obtained by PING operation from a client in Nanjing University to eight web servers located in American and UK universities, CAM_AC, CMU,

With regard to NLMS algorithm, its precision is slightly improved by 0.1 dB on average, but the algorithm complexity increases more than 90%, as shown in Table 2.

Table 1 PSER (dB) of AR, DIAR and NLMS methods CAM_AC

CMU

Columbia Harvard

MIT

Stanford

USC

Washington

AR

31.930

20.237

17.286

20.137

22.964

8.027

18.396

20.431

NLMS

35.698

30.664

26.890

27.331

27.869

15.847

28.085

25.080

DIAR

35.713

30.703

27.035

27.403

27.937

16.143

28.157

25.183

1352 & The Institution of Engineering and Technology 2010

IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

www.ietdl.org Table 2 Computations of DIAR and NLMS methods Algorithm Variables number

Operations number

where Ndownflow represents the number of underflow packets whereas Nupflow represents the number of overflow packets. The total number is Ntotal .

Division Multiplication DIAR/AR

3

0

3

NLMS

38

18

93

4.3 Testing of the new proposed algorithm Firstly, the proposed new-AMP algorithm employs DIAR to predict packet delay, which is shown with better performance in Section 4.2. For the purpose of comparing the performance of the proposed new-AMP with AMP-live, it is assumed that end-to-end delay is 160 ms, video sample frame rate is 25 frames/s and maximum tolerance of playout delay is 500 ms. As a result, the initial parameters are set as below

4. Another proposed performance metrics is bdave , which is the mean delay of the receiver buffer

bdave =

(

N

i=1(tp,i

− tr,i ))

N

where N is the number of the packet series. The simulation results are shown in Figs. 5 and 6 with the detailed data included in Table 3. It can be seen that compared with non-AMP, the mean buffer delay (bdave) is decreased by 25% with AMP and 50% with new-AMP. For the mean of overflow/underflow probability Pdrop , there is about 22% increase with AMP and a slightly decrease with new-AMP compared with non-AMP.

Interval = 40 ms Nmax = 6 Nmin = 3 The maximum size of buffer in normal status is Nmax ∗ Interval + 160 ms = 400 ms, which is less than the delay tolerance 500 ms. The proposed new-AMP algorithm consists of four steps as follows: 1. The adjustment of playout speed is based on the thirdorder predicted values in the approach of adaptive playout algorithm, that is when the ith packet is received, the i + 3 packet’s delay and buffer state is used to adjust the playout speed.

Figure 5 Pdrop of methods non-AMP, AMP and new-AMP

2. To maintain the mean end-to-end delay of 160 ms, the initial network delay series must be normalised. The normalised parameter can be determined by f ¼ 160/m, where u is the mean of the series. 3. The performance of playout algorithm is assessed by the probability of overflow/underflow Pdrop and mean buffer delay bdave . If and only if tp,i , tr,i , a packet is judged as underflow, that is all packets are discarded because of the packet late arrival to miss the scheduled time. If and only if tp,i − ts,i . 500 ms, a packet is judged as overflow, that is packet is discarded because the delay tolerance is reached. In order to simplify the simulation, the performance metrics are calculated as Pdrop =

(Ndownflow + Nupflow ) Ntotal

IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

Figure 6 BDave of methods non-AMP, AMP and new-AMP 1353

& The Institution of Engineering and Technology 2010

www.ietdl.org Table 3 Performance of non-AMP, AMP and new-AMP algorithms Amount of packet

Non-AMP

AMP

New-AMP

Over/ under flow packet

Pdrop (%)

bdave (ms)

Over/ under flow packet

Pdrop (%)

bdave (ms)

Over/ under flow packet

Pdrop (%)

bdave (ms)

CAM_AC

251 151

267

0.11

199.52

596

0.24

156.60

325

0.13

112.29

CMU

91 678

416

0.45

219.60

736

0.80

156.85

498

0.54

79.74

Columbia

43 234

557

1.29

221.59

582

1.35

156.48

560

1.30

127.59

Harvard

90 748

404

0.45

205.83

565

0.62

157.12

500

0.55

84.52

MIT

84 104

126

0.15

193.72

150

0.18

156.92

122

0.15

87.60

Stanford

91 182

2406

2.64

182.16

2423

2.66

158.69

1645

1.80

97.66

USC

91 729

419

0.46

210.38

793

0.86

157.12

451

0.49

104.31

Washington

247 327

147

0.06

217.98

368

0.15

156.61

296

0.12

74.40

0.70

206.35

0.86

157.05

0.64

96.01

average

In addition, the number of underflow packets exceeds the number of overflow packets in non-AMP, whereas the number of overflow packets exceeds the number of underflow packets in AMP and new-AMP.

5

Conclusions

In this paper, an improved AMP scheme is proposed. Improvements are presented in terms of smoother playout rate adjustment, considering both packet lifetime and delay prediction. The proposed new-AMP algorithm adjusts the packet playout speed based on the prediction of channel and buffer status, thus providing more robust and smoother media playout. Moreover, the refined adjusting step in playout speed and the consideration of the maximum endto-end delay improve the efficiency and effectiveness further. Simulations have confirmed performance improvements of the proposed method. Compared with non-AMP algorithm, the packet delay is decreased by 50% with a lower packet loss rate because of over/under flow. Compared with AMP-live method, new-AMP not only decreases the packet delay by 40%, but also reduces the packet loss rate. By use of a differential AR algorithm, the proposed method effectively removes the interdependency in the delay sequence. In this way, the delay distribution of practical packet arrival streaming is approaching to N(m, s2), which satisfies the assumption of ARMA model and thus results in satisfactory prediction precision. The performance metric is packet loss rate at application layer, resulting from network congestion, physical channel impairments or missing the scheduled playout time. For wireless channels, because of the inevitable channel impairment, the delay jitters is fluctuating wider and the 1354 & The Institution of Engineering and Technology 2010

buffer status is changing frequently. It will influence the prediction result and make the playout speed adjustment triggered frequently accordingly. To further improve the algorithm having better adaptation to the wireless environment, it may be needed to distinguish the buffer status change because of physical impairments from that due to network congestion and employ different scheme for them. A lazy adaptation strategy with auxiliary compensation can be figured out for the wireless case to avoid unnecessary playout speed adjustment.

6

Acknowledgments

This work is financially supported by NSFC Project No. 60872018, 973 project No. 2007CB310607 and SRFDP project No. 20070293001

7

References

[1] KALMAN M. , STEINBACH E., GIROD B.: ‘Adaptive playout for real-time media streaming’. Proc IEEE ISCAS’02, Scottsdale, AZ, May 2002, vol. 1, pp. 45– 48 [2] STOCKHAMMER T., JENKAC H., KUHN G.: ‘Streaming video over variable bit-rate wireless channels’, IEEE Trans. Multimed., 2004, 6, (2), pp. 268 – 277 [3] FUJIMOTOL K., ATA S., MURATA M.: ‘Adaptive playout buffer algorithm for enhancing perceived quality of streaming applications’, ACM Telecommun. Syst., 2004, 25, (3), pp. 259– 271 [4] TAO D., HOANG H., CAI J.: ‘Optimal frame selection with adaptive playout for delivering stored video under constrained resources’. Proc. IEEE ICME 2007, pp. 1798 – 1801 IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

www.ietdl.org [5] STEINBACH E., FARBER N., GIROD B.: ‘Adaptive playout for low latency video streaming’. Proc. IEEE Int. Conf. on Image Proc., October 2001 [6] CHUANG H.C. , HUANG C.Y. , CHIANG T.H.: ‘Content-aware adaptive media playout controls for wireless video streaming’, IEEE Trans. Multimed., 2007, 9, (6), pp. 1273 – 1283 [7] YANG Y.-H., LU M.-T., CHEN H.H.: ‘Smooth playout control for video streaming over error-prone channels’. Proc. Eighth IEEE Int. Symp. Multimedia, San Diego, CA, 2006, pp. 415–418 [8] LI Y., MARKOPOULOU A., APOSTOLOPOULOS J., BAMBOS N.: ‘Content-aware playout and packet scheduling for video

IET Commun., 2010, Vol. 4, Iss. 11, pp. 1348 – 1355 doi: 10.1049/iet-com.2009.0467

streaming over wireless links’, IEEE Trans. Multimed., 2008, 10, (5), pp. 885– 895 [9] KALMAN M., STEINBACH E., GIROD B.: ‘Adaptive media playout for low delay video streaming over error-prone channels [J]’, IEEE Trans. Circuits Syst. Video Technol., 2004, 14, (6), pp. 841– 851 [10] KALMAN M., STEINBACH E., GIROD B.: ‘Adaptive playout for real-time media streaming [J]’. Proc. IEEE ISCAS’02, 2002, vol. 1, pp. 45– 48 [11] JIAO L., ZHANG D.E., BI H.: ‘Differencing AR algorithm for packet delay prediction [J]’, Prog. Nat. Sci., 2006, 4, (16), pp. 434– 440

1355

& The Institution of Engineering and Technology 2010