MobSOS - A Testbed for Mobile Multimedia Community Services

MobSOS - A Testbed for Mobile Multimedia Community Services Dominik Renzel, Ralf Klamma, Marc Spaniol RWTH Aachen University Chair of Computer Science 5 Ahornstr. 55, D-52056 Aachen, Germany {renzel|klamma|spaniol}@i5.informatik.rwth-aachen.de

Abstract Due to recent developments in the domain of mobile broadband communication and a growing availability of standardized development tools for mobile devices a new generation of mobile services and applications is expected to emerge in the near future combining well-established multimedia and community concepts with mobile aspects. However, it is often challenging to predict the success of such new services. In this paper we present a short overview of the MobSOS testbed and its homonymous success model based on a combination of traditional information system success models with modern requirements for mobile multimedia communities. Finally we outline the application of the MobSOS testbed to NMV Mobile, a mobile multimedia capturing, annotation and retrieval tool.

1. Introduction Recent developments in information and communication technology exhibit a rapid increase of web-based information systems providing mobile services, especially in the domain of multimedia technology. On top of plain content delivery multimedia metadata standards like MPEG-7 [9] or MPEG-21 [8] already paved the way for a variety of new services. An additional key success component of such services is and will be the support for online communities, thereby introducing social concepts to allow people to meet and interact on the web. In connection with the availability of wireless broadband connectivity and standard developer support in future mobile devices the above concepts become combinable with mobile context aspects, thus providing users with a ubiquitous mobile multimedia community experience. In the context of the UMIC excellence cluster, one central research interest is on the measurement or even the prediction of the success of such new generation services. In this paper we introduce MobSOS, a testbed for rapid prototyping, testing and the quality evaluation of

mobile multimedia community services based on the new MobSOS success model. This paper is structured as follows. First, the MobSOS testbed is introduced step by step. Its conceptional core built by the MobSOS success model for mobile multimedia community services is described in section 2. The implementation of the MobSOS testbed is described in section 3. In section 4 we outline the application of the MobSOS testbed on an example usecase service of MPEG-7 based automatic image geotagging. Finally, we conclude with an outlook to future work in section 5.

2. The MobSOS Success Model The first important task was to create an appropriate model for measuring the success of current mobile multimedia community services. Thus, we conducted an extensive literature research in the domain of information system (IS) success models over the last three decades. In this section we first describe early research in the domain of IS success. Then, we point out missing aspects of early considerations and introduce additional success factors to be included into an up-to-date success model for mobile multimedia community services. Finally, we briefly describe the design and validation process of the MobSOS success model. One of the early findings was that measuring IS success is a multidimensional problem and therefore must be approached accordingly [12, 4]. During the 1980s and 90s the main focus was clearly on measuring the success of management information systems. In 1992 DeLone and McLean released a seminal contribution with their D&M IS success model [3] based upon extensive literature research of over a hundred empirical and conceptual studies on managing IS success. In addition to a plain taxonomy consisting of six major success dimensions they also pointed out causal interrelationships among dimensions. In the context of this work the D&M IS success model was chosen to serve as a foundation framework due to its high acceptance and proven validity among researchers. Although most of

LAS API MobSOS Quality Analysis Management

LAS Community Management

Service 1 method 11(...) method 12(...) ...

LAS Element Management

...

Quality Statistics Database

MobSOS Analytics Service

MobSOS User Survey Management

Service n

MobSOS Feedback Service

method n1(...) method n2(...) ...

MobSOS Communication Monitor

LAS Object Management LAS Security Management

LAS Session Management LAS HTTP Connector

Server Client

LAN, WLAN, UMTS, GSM

LAS HTTP Client (regular JDK)

LAS HTTP Client (Java ME, CDC)

LAS HTTP Client (Java ME, CLDC)

Desktop/Laptop PCs ...

PDAs, Pocket PCs ...

SmartPhones, Mobile Phones

Figure 1. MobSOS Testbed Extensions on the LAS (Lightweight Application Server) Architecture DeLone and McLean’s success measures are generic and universally valid to also apply to current systems, there are modern mobility, multimedia or community aspects that are not covered. More recent work in the domain of mobile services showed that among others concepts like Quality of Service [2] or Mobile Context [5, 10] are considered as highly important, although not mentioned in [3]. More recent work on community system success by Leimeister et al. [7] introduced community system success factors and revealed interesting results like significant differences in success criteria importance weighting among different populations or a high demand for security on handling member data. Thus, it became necessary to augment the wellestablished D&M IS success model by a set of new factors and measures, in total capturing the success of current information systems. Over 140 possibly relevant factors were collected, classified to the six D&M success dimensions and reduced to an initial preselection list of success factors. Thereby each factor was required to fulfill the following constraints: • Factor is considered as highly relevant for mobile multimedia community service success, either because it is universally valid or it is expected to play a key role in the domains mobility, multimedia and communities. • Factor exhibits a minimal amount of ambiguity in meaning. • Measures for factor are capturable by the MobSOS testbed, i.e. are either machine-accessible and computable using the MobSOS monitoring support or acquirable in questionnaires with a minimal amount of unambiguous items using the MobSOS survey support (see section 3).

All other factors, i.e. potentially outdated or irrelevant factors not specific to current mobile multimedia community services were dropped. The initial design is likely to be subject of improvement after one or more model calibration iterations in future field studies. It must be noted that such preselection steps introduce the potential risk of adopting irrelevant respectively abandoning relevant factors. In a first pre-evaluation phase an initial set of sample service quality measurement data was used to test the feasibility of applying established multivariate statistical techniques such as factor and multiple regression analysis in order to gain knowledge if the model really measured service success and - if so - what factors were expected to significantly contribute to success. In this early stage the primary focus was on the pre-calibration of the success model, and not the quality evaluation of test services. In the current development stage we concentrate on the definition of measures for all preselected success factors. An overview of the MobSOS success model is available at http://www.multimedia-metadata. info/Events/m3r08/mobsos/addendum1.

3. The MobSOS Testbed For the actual implementation of the MobSOS testbed, we extend the LAS community engine already presented in [11] by a set of software components that in entirety enable the engine to capture, store and evaluate measures of the underlying success model. This section introduces a set of LAS extensions towards the MobSOS testbed for mobile multimedia community services. Figure 1 describes the extended architecture. All elements depicted with a black font represent existing parts

of the architecture being further described in [11]. All elements depicted with a white font represent the actual extensions proposed in the scope of this work. The MobSOS communication monitoring module is designed to capture communication log data that can later on be combined to higher-level measures serving as proxies for success. Therefore the MobSOS communication monitoring module hooks into LAS communication and records a set of data fields containing information on requests and their responses, occurring exceptions and mobile context information transmitted with each request. Monitoring data is then logged to a database to be used for later statistical analysis, either by the MobSOS quality analysis module or with standard statistics tools such as SPSS. Detailed information on all log data fields recorded by the MobSOS monitoring module is available at http://www.multimedia-metadata. info/Events/m3r08/mobsos/addendum2. The MobSOS user survey support module is designed to capture data reflecting all those factors from the success model that cannot be acquired by monitoring. Capturing data mainly consists of surveying system users with the help of questionnaires. The user survey management core generates questionnaires, possibly adapted to the preferences of different individuals resp. communities or to particular service functionality. Additionally, it stores questionnaire response data to the statistics database appropriately. A MobSOS feedback service provides users with support for filling in and submitting questionnaires. The MobSOS quality analyis & presentation module is designed to analyze and present the data collected in the quality statistics database. Its main purpose is to transform the rather low-level raw data captured by both communication monitoring and user surveys into high-level quality information reflecting the success model. The other part of this module is designed to present service quality information offered by the system to users in an appropriate form including means for presentation filtering (e.g. by users, communities, date, etc.). A MobSOS analytics service provides the presentation functionality offering users an interface to the analysis core element and the database. However, the quality of the success estimation is determined by both the validity of the success model and the number of samples acquired during evaluation. Together, monitoring and survey response data constitute one data record acting as input for further service success analysis.

4. MobSOS Service Evaluation Applied This section gives an outline of how the MobSOS testbed can be applied in order to analyze a set of mobile multimedia community services in the scope of a field study. The evaluation of service success is mainly dependent on the

measurement data generated by evaluators. In general, this part is conducted by each evaluator in a two-step approach. The first step is to actively, however voluntarily use the service under consideration on one or more mobile client devices, while communication is monitored by MobSOS. The second step is to fill in questionnaires using the MobSOS feedback service. As an example evaluation scenario we introduce a mobile version of the multimedia capturing, annotation and retrieval tool NMV[6] as a mobile application that a) demonstrates the power of incorporating mobile context information into new applications and b) is powered by a set of LAS MPEG-7 multimedia services to be evaluated by the MobSOS testbed. Figure 2 shows a design study sample of the NMV Mobile interface on a Nokia N95 8GB. After hav-

Figure 2. NMV Mobile Design Study ing registered for the evaluation process an evaluator E logs into NMV Mobile on a preconfigured mobile device using his LAS community account. While he captures a photo with the mobile device’s integrated camera NMV automatically records geotagging data by using current mobile context information available from the device’s built-in GPS receiver (i.e. location and time). After having decided to upload the photo to a central media repository E uses NMV to add annotations in form of freetext, keywords and semantic base information and finally registers the photo. NMV then merges manual annotations and automatic geotagging data and invokes a set of LAS MPEG-7 service methods for generating and storing MPEG-7 compliant multimedia content resp. semantic basetype descriptions in a backend XML database. Afterwards E submits a search expression to find other photos. NMV mobile again makes use of MPEG-7 service functionality by invoking respective media search methods. Then E quits his NMV Mobile session. On the server side MobSOS has logged all invocation requests and respective mobile context information transmitted with each request. The evaluator then proceeds to fill in a questionnaire covering all LAS service functionality he has used so far (i.e. register media, media retrieval, define semantic base information). With an appropriate amount of mea-

surement data service success evaluation can finally be executed. The MobSOS quality analysis & presentation module provides a set of statistics computed with the same techniques initially used for the MobSOS model precalibration. Multiple regression analysis will reveal regression coefficients, model goodness-of-fit statistics, overall model and individual measure success relation significance statistics, correlations among individual measures, as well as basic descriptive statistics such as means, variances, deviations, etc. These statistics will provide valuable information on service quality in all success model dimensions, e.g. which factors were most influential for the success of a service. Finally, this information can be used for predicting effects on service success if improvements on important success factors were implemented.

5. Conclusions & Outlook To the best of our knowledge there is no other information system success model for mobile community information systems (MCIS). After a strong research period in the 1980s and 90s a parallel branch of research was introduced for studying the success of online communities. In our MobSOS model we join both branches again to recreate a comprehensive MCIS success model. The model is currently under development. Creating common testbeds for the evaluation of mobile multimedia services based on MPEG-7 and MPEG-21 is one of the major tasks of the Multimedia Metadata Community (http://www. multimedia-metadata.info). As we have already developed and deployed a couple of MPEG-7 and MPEG21 services in different tools of the community future work will include the systematic evaluation of these services in advanced scenarios of distributed mobile multimedia information systems such as the Virtual Campfire Scenario [1]. Virtual Campfire is a mobile social software for cross-media communities. In the moment the target community are professionals, researchers and cultural bureaucrats preserving the cultural heritage of Afghanistan, especially the Bamiyan valley and its Buddha statues.

Acknowledgements This work is supported by the Excellence Initiative of German National Science Foundation (DFG) within the research cluster “Ultra High-Speed Mobile Information and Communication” (UMIC).

References [1] Y. Cao, M. Spaniol, R. Klamma, and D. Renzel. Virtual Campfire - A Mobile Social Software for Cross-Media Communities. In K. Tochtermann, H. Maurer, F. Kappe, and

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

A. Scharl, editors, Proceedings of I-Media’07, International Conference on New Media Technology and Semantic Systems, J.UCS (Journal of Universal Computer Science) Proceedings, pages 192–195, Graz, Austria, September 5–7 2007. Springer-Verlag. D. Chalmers and M. Sloman. Survey of Quality of Service in Mobile Computing Environments. Research Report 98/01, Department of Computing, Imperial College London, 1998. W. H. Delone and E. R. McLean. Information Systems Success: The Quest for the Dependent Variable. Information Systems Research, 3(1):60–95, 1992. P. Ein-Dor and E. Segev. Organizational Context and the Success of Management Information Systems. Management Science, 24(10):1064–1077, June 1978. M. Kakihara and C. Sørensen. Mobility: An extended perspective. In Proceedings of the 35th Hawaii International Conference on System Sciences (HICSS), pages 1756–1766, January 2002. R. Klamma, M. Spaniol, and D. Renzel. Community-Aware Semantic Multimedia Tagging From Folksonomies to Commsonomies. In K. Tochtermann, H. Maurer, F. Kappe, and A. Scharl, editors, Proceedings of I-Media’07, International Conference on New Media Technology and Semantic Systems, J.UCS (Journal of Universal Computer Science) Proceedings, pages 163–171, Graz, Austria, September 5–7 2007. Springer-Verlag. J. M. Leimeister, P. Sidiras, and H. Krcmar. Success factors of virtual communities from the perspective of members and operators: An empirical study. In HICSS ’04: Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS’04) - Track 7, page 70194.1, Washington, DC, USA, 2004. IEEE Computer Society. Moving Picture Experts Group. (ISO/IEC JTC1/SC29/ WG11/N5231:2002) - MPEG-21 Overview, 5th edition, October 2002. Moving Picture Experts Group. (ISO/IEC 15938-5:2003) - The MPEG-7 Multimedia content description interface Part 5: Multimedia description schemes, 1st edition, May 2003. N. Roy, H. Scheepers, E. Kendall, and A. Saliba. A Comprehensive Model Incorporating Mobile Context to Design for Mobile Use. In CHI-SA 2006 - Proceedings of the 5th Conference on Human Computer Interaction in Southern Africa, pages 22–30, January 2006. M. Spaniol, R. Klamma, H. Janßen, and D. Renzel. LAS: A Lightweight Application Server for MPEG-7 Services in Community Engines. In K. Tochtermann and H. Maurer, editors, Proceedings of I-KNOW ’06, 6th International Conference on Knowledge Management, Graz, Austria, September 6–8, 2006, J.UCS (Journal of Universal Computer Science) Proceedings, pages 592–599. Springer-Verlag, 2006. E. Vanlommel and B. D. Brabander. The Organization of Electronic Data Processing (EDP) Activities and Computer Use. The Journal of Business, 48(3):391–410, July 1975.