EATIS 2007
A Mobile Audio Messages Streaming System MABEL VAZQUEZ National Institut of Telecommunications, GET-INT/ Autonomous University of Baja California 9, rue Charles Fourier- 91011 Evry FRANCE / MEXICO
[email protected] [email protected]
PIERRE VINCENT MobKit 9, rue Charles Fourier- 91011, Evry FRANCE
[email protected]
Abstract In this paper we describe a mobile audio messages streaming system, whose aim is to create and deliver streaming of voice messages to mobile phones. Streaming is a very attractive multimedia service that has many applications. It allows users to open a connection with a server to receive media and start to play it as soon as sufficient information has arrived. As mobile phones have different capabilities, it has not been easy to develop a streaming application that fits most mobile devices. This project presents a solution to implement this application in a way to reach most mobile phones even if originally they lack of features to allow streaming.
2. System Description The main goal of the system is to stream voice messages to mobile phones with Java 2 Micro Edition (J2ME) support, using the Real Time Streaming Protocol. Voice messages are also generated by the J2ME clients using their Mobile Media API (MMAPI) capabilities. MMAPI is an extension to J2ME. It supports time-based multimedia on small wireless devices [2]. Figure 1 shows the main components of the system as well as the protocols involved.
Keywords: Mobile computing, streaming, java, multimedia.
1. Introduction One of the most attractive mobile multimedia services is streaming. Streaming is the process of transferring data via a channel to its destination, where it is decoded and consumed via the user or device in real time, that is, as the data is being delivered. It differs from non-streaming processes because it doesn’t require the data to be fully downloaded before it can be played. This technique frees up terminal memory, and could also allow for media to be sent live to clients as the media event happens. As streaming is not the property of the data being delivered, but an attribute of the distribution channel, technically, most media can be streamed [1]. The Real Time Streaming Protocol (RTSP) provides the means to control the transport of the streaming media in a network and is the most widely used streaming protocol. Some new mobile phones provide support for RTSP but the majority of devices still do not support this protocol. In this paper we present a new system that provides audio messages streaming to mobile phones using RTSP even if formerly the mobile phone does not support this protocol.
Figure 1. System architecture
The PHP server in the system is used to administrate the voice messages access. This server accepts only messages from authorized members, which are users that are already registered in the database. Once an audio file is received, the PHP server stores it in the space corresponding to the destination member and updates the related received voice messages list. When a J2ME client desires to access its messages, it must first access the PHP server to obtain the corresponding messages list where the locations of the audio files are registered. This information is sent to the
EATIS 2007
RTSP server when the client starts a session to play the corresponding audio file. All J2ME mobile phones in the system must have the capability to capture audio. The system allows WAV or Adaptive Multi-Rate (AMR) audio formats. The audio message is then transmitted to the PHP server using the Hypertext transfer protocol (HTTP). The streamed audio messages are in AMR format because AMR has been chosen by the Third Generation Partnership Project (3GPP) as the mandatory codec for third generation cellular systems [3]. AMR has also a grater compression rate than other encoding technologies like WAV and MP3. In the next section the AMR audio format and the main functions of each component in the system are described.
2.1. AMR Audio Format The AMR codec is a multi-mode codec that supports 8 narrow band speech encoding modes with bit rates between 4.75 and 12.2 kbps (Modes 0 to 7). Each AMR audio frame consists of 1-byte header, and then the rest of the frame is audio data. The entire frame is fed into the AMR decoder. The frame size can be deduced from the frame header. The top 4 bits of the header comprise the CMR (Codec Mode Request) which is set to the frame type indicator index or AMR mode. The AMR audio format is slightly different when streamed as payload within a RTP (Real Time Protocol) packet than when played as an audio file. AMR in storage format is used for storing AMR or AMRWB (Wide-Band) speech frames in a file or as an e-mail attachment. Each AMR file consists of a 6-byte header that identifies the file as AMR audio. A single channel AMR file header contains only a magic number. This magic number consists of the ASCII character string “#!AMR\n” (or 0x2321414d520a in hexadecimal). Each AMR audio file is then packed frame by frame. Figure 2 shows the general structure of an AMR file. Header File (#!AMR\n) Header 1 Speech Frame 1 …… Header n Speech Frame n Figure 2. General structure of an AMR file
When AMR audio is sent as RTP payload, its structure differs from AMR files, since the list of headers is followed by the audio frames as depicted in figure 3. Header 1
Header 2 Header 3 Frame 1 Frame 2 Frame 3 Frame 4
Header 4
Figure 3. AMR as RTP compound payload
Figure 3 shows a compound payload with several AMR frames nevertheless a RTP packet may contain also only one AMR frame as payload. The sampling frequency used in AMR is 8000 Hz and the speech encoding is performed on 20 ms speech frames. Therefore each encoded AMR speech frame represents 160 samples of the original speech [4].
2.2. J2ME Client The main operations of a J2ME client in this system are: Set the user member; create a new voice message; get the voice messages list and listen the voice messages. A member is an authorized user that is already registered in the database and that may access the system. Each declared member has a dedicated space of safeguard that is used to maintain an inbox. All messages from other members are stored in the inbox and the member may access it from any mobile. After the inbox is recovered, a list of all audio messages is maintained locally using the device’s Record Management System (RMS), this allows the member to communicate with the RTSP server and provide it with the location of the audio file that wants to access. To generate a new message the phone uses the MMAPI audio capture feature. Most mobile phones have only the capacity to capture WAV audio, but recent models can also capture AMR audio, which is a most compressed and efficient format. The message is then transmitted to the server using HTTP. All messages are stored in the server as AMR files; if the phone is capable of capture only WAV format audio, the PHP server transforms the audio file to AMR mode 2. Most phones are able to play AMR but not to capture audio in this format, for that reason the system supports both formats. To receive messages the client first connects to the PHP server using HTTP. The PHP server maintains the record of all audio messages received and sends them as a list to the client. When the client wants to listen a message it
EATIS 2007
must connect to the RTSP server. The RTSP server starts a session with this client and the message is streamed using RTP. Each AMR file is streamed frame by frame as RTP payload using a Datagram Connection. Each datagram consists in the RTP header and one AMR frame as payload. The structure of an RTP packet header is shown in figure 4.
Figure 4. RTP header
In principle J2ME and MMAPI don’t support playback of streaming media. The Player in MMAPI requires a whole AMR audio file to be able to play it. To resolve this aspect it was necessary to construct new smaller files from the original one. The server streams one AMR frame from the whole file in each RTP packet. The client waits to have a specified amount of AMR frames to construct an AMR file from the RTP packets payload with the format depicted previously in figure 2. While this small file is played, the server continues sending RTP packets, for this reason two buffers and two players where used. When an AMR file is completed, the first player starts to play it and the second buffer starts to receive the AMR frames, then when the second buffer is filled, the second player starts to play this buffer and the first buffer starts to receive the AMR frames. This process is done during all the transmission and is depicted in figure 5.
Figure 5. Streaming in a J2ME device
The J2ME client is able to send SETUP, PLAY, PAUSE and STOP events to the RTSP server to control the voice message transmission. RTSP requests are implemented using a Socket connection while RTP communication is done using a Datagram connection; as a result it is not indispensable that the phone supports RTSP.
2.3. PHP Server The PHP Server was implemented using PHP version 5.0, Apache Server version 2.0 and MySQL version 5.0. The PHP scripts in this server accomplish the following functions: -To authenticate the users that want to send and receive messages. Communication with the MySQL database is required for this function. -To receive the audio messages and store them in the directory corresponding to the authenticated user. -To generate the list with all the messages that will be sent to the client when it requires it. -To convert from WAV to AMR audio format. -To delete voice messages files when the J2ME client deletes them from its messages list. All authorized members must already be registered in the MySQL database; this function can be done using an HTML administration tool developed for this purpose.
2.4. RTSP Server The Real Time Streaming Protocol, or RTSP, is an application-level protocol for control over the delivery of data with real-time properties. RTSP provides an extensible framework to enable controlled, on-demand delivery of real-time data, such as audio and video. This protocol is intended to control multiple data delivery sessions, and provide a means for choosing delivery channels such as UDP, multicast UDP and TCP, as well as for choosing delivery mechanisms based upon RTP [5]. RTSP may be sent over TCP while the media streams normally use UDP as the transport mechanism. Thus, the continuity of the media stream is not affected by delays in RTSP signalling. The RTSP request is a signalling message from the client to the server. The server sends responses back to the client by RTSP response status codes that are mainly reused from HTTP. In this system the RTSP server was implemented with Java SDK 1.5. The server establishes a RTSP session when a J2ME client requests a voice message. The RTSP protocol is used to set up the data stream between the server and the application on the mobile phone. It is also used to handle the PLAY, PAUSE and STOP events. Once the session is established, RTP is used to deliver the actual data that is streamed. The SETUP event causes the server to allocate resources for a stream and start a RTSP session. The SETUP request from the client to the server includes the transport mechanism to be used for the streamed media as well as
EATIS 2007
the audio file location. PLAY starts data transmission on a stream allocated via SETUP. PAUSE temporarily halts a stream without freeing server resources, and TEARDOWN (STOP in the J2ME Client) frees resources associated with the stream. Figure 6 shows the RTSP session initiation procedure.
3. Results The system was tested using the Sprint simulator with emulators for the Samsung A900, A920, A940 as well as Sanyo MM-7500 and 9000 mobile phones. These phones are able to play AMR files but can capture only WAV audio. The main problem with this lack of capacity to capture AMR format is the limited audio time that can be captured. After many tests this time couldn’t exceed 15 seconds in these phones. For longer messages an Out of memory Error was obtained when trying to send the message using HTTP. A WAV file of 15 seconds has a size of 228 kB while an AMR file of same time is less than 20 kB long. We tested the J2ME client using also the Siemens SL65 emulator, which has the AMR audio capture feature. With this phone there weren’t errors, even sending messages of several minutes. To receive streaming messages using the RTSP server the J2ME client works fine in both emulators, but in the SL65 there is a delay when changing from one buffer to the other. This delay is not present in the Sprint emulator where the change is almost imperceptible.
instead. HTTP and RTSP are both application-level protocols that allow remote retrieval of data. However streaming content over HTTP is not very efficient because it is based on the Transmission Control Protocol (TCP), which includes delays in media delivery since it requires delivery acknowledgment. On the other hand, when using RTSP, media is transmitted over RTP which is based on the User Datagram Protocol (UDP) that ensures faster delivery. Besides, RTSP offers control mechanisms to access the media data like PLAY, PAUSE and STOP while HTTP doesn’t. The system presented in this paper implements RTSP and RTP in the server as in the J2ME clients, therefore is a much efficient streaming solution compared to HTTP streaming.
5. Conclusions and Future Work The mobile messages streaming system presented in this article provides an efficient solution for streaming audio using RTSP to J2ME mobile phones, even if originally they do not support this protocol. Streaming has many applications and we consider that the system can be extended to support other services and applications like video streaming and video conferences. We plan also to extend the system to allow multicast communications. Another important aspect is that, formally, J2ME doesn’t support streaming because it needs to play an entire AMR format file. In this system it was possible to implement streaming using 2 buffers and two players, the delay is minimal but maybe it could be improved using more buffers. As future work it is essential to test the system using real devices as well as evaluate its performance and security.
6. References [1] Montes Hector, et al., “Deployment of IP Multimedia
Figure 6. RTSP session initiation procedure
4. Related Work As most mobile phones don’t support RTSP there are other implementations of streaming using HTTP [6]
Streaming Services in Third-Generation Mobile Networks”, IEEE Wireless Communications, Oct-2002, pp 84-92. [2] Mobile Media API 1.2. Sun. http://java.sun.com/products/mmapi/ . [3] 3GPP TS 26.09 “Adaptive Multi-Rate (AMR) speech transcoding”, 3rd Generation Partnership Project (3GPP), march 2003. [4] Sjobern Johan, et al. “RTP Payload and File Storage Format for the Adaptive Multi-Rate (AMR) and Adaptive Multi-Rate Wideband (AMR-WB) Audio Codecs”, RFC 3267, August 2006. [5] H. Schulzrinne, A. Rao, R. Lanphier and M. Westerlund, “Real Time Streaming Protocol”, RFC 2326, March 2003. [6] Baumgart Andre, et al., “A Platform-Independent Adaptive Video Streaming Client for Mobile Devices”, International Conference on Mobile and Wireless Communications, sept.1921, 2005, Morocco.
