Professional Documents
Culture Documents
Maiju Vuolle
Department of Business Information Management and Logistics Tampere University of Technology P.O.Box 541 33101 Tampere, Finland +358 40 559 6424
Titti Kallio
TeliaSonera P.O.Box 585 00051 Sonera, Finland +358 40 503 6957
Minna Kulju
VTT, Media and Mobile usability Technical Research Centre of Finland P.O. Box 1300 33101 Tampere, Finland +358 20 722 3386
titti.kallio@teliasonera.com
Heli Wigelius
Department of Software Systems Tampere University of Technology P.O. Box 589 33101 Tampere, Finland +358 40 500 6104
mari.tiainen@nokia.com ABSTRACT
teija.vainio@tut.fi
In this paper, three dimensions are conceptualized to represent elements of mobile business service experience. By combining these perspectives, namely usability, mobile working context and mobile work productivity, we aim to understand the nature of mobile work and how mobile business services could support users in this context. A questionnaire, MoBiS-Q, for measuring these dimensions has been developed and tested in three pilot studies during real service development processes. Iterative item generation and refinement were conducted through examination of the literature, interviews and pre-testing. MoBiS-Q is a multidisciplinary tool that provides a basis for joint development between relevant parties and departments in an organization, including user representatives, sales, marketing, product management, technology, and usability.
heli.wigelius@tut.fi
1. INTRODUCTION
Mobile business services are increasingly being used for work purposes in several domains, such as the police [17], independent consultants [20], field force [35], sales and marketing [44]. These different activities have various requirements for both mobility and services supporting mobile work tasks [5]. In this paper, the concept of a mobile business service refers to a business-toemployee service that the user operates using a portable device. Examples of such services include business data services that provide access to corporate intranet and email, and services that enable data collection in the field (see, e.g., [35]). In order to develop successful mobile business services, there is a need to understand the dimensions of success. In earlier studies, perceived ease of use and perceived usefulness have been emphasized as the most critical factors for technology acceptance in the work context [6]. In addition, the usability of a mobile business service has been seen as an important driver of the business impacts of using the service [31]. However, examining the customer experience of mobile business services is challenging because user experience consists of many elements, not only technology. In addition, in the area of business services, to be a critical factor, user experience needs to be combined with enhancements in work productivity. A major challenge for improvements here is also that customer experience is dependent on a network of technology and service providers. There is no solid way to evaluate if a mobile business service is appropriate both from a perspective of usability and productivity impacts. Conducting traditional usability and productivity evaluations for mobile business services increases the risk of irrelevant results by failing to consider the context of mobile work. Thus, this paper focuses on the evaluation of experience including usability and productivity factors for mobile business services. The purpose of this paper is to investigate the dimensions that determine users mobile business service
General Terms
Measurement, Human Factors, Performance, Economics, Management
Keywords
Evaluation, Questionnaire, Usability, Mobile work context, Productivity, Mobile business service
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MobileHCI 2008, September 25, 2008, Amsterdam, the Netherlands. Copyright 2008 ACM 978-1-59593-952-4/08/09$5.00.
FP
53
experience and to develop a questionnaire for measuring that experience. This questionnaire is meant for service developers for benchmarking and evaluating the success of the service against competitors, for evaluating the new versions of the service, and for evaluating the benefits the service could provide for end users and companies deploying the service. While the focus of this paper is to develop and evaluate a specific tool, our findings will also be of interest to those concerned with evaluation of usability and productivity within mobile business services in general. The primary contribution of this paper is to provide a novel multidisciplinary tool for evaluating key usability and productivity issues based on individual workers subjective views on these issues. The evaluation tool is called mobile business service questionnaire (MoBiS-Q). To develop the questionnaire, we iteratively generated and refined items based on the literature, existing case studies ([30]; [31]), and interviews. In order to evaluate MoBiS-Q, we conducted three pilot tests in three different companies with service developers and real end-users. A secondary contribution of this paper is to emphasize what kind of challenges there are in integrating the evaluation results in the actual development process of a mobile business service. The paper proceeds as follows: the following section provides some background for measuring usability and impacts in general and of earlier studies of mobile commerce and systems. Section 3 focuses on the evaluation of mobile business service experience and describes the concept and dimensions of mobile business service experience in more detail. Section 4 presents the iterative development of our evaluation tool MoBiS-Q, and Section 5 outlines our findings. The final section concludes.
of mobile applications. These challenges include, for example, mobile context, multimodality, connectivity, small screen size, different display resolutions, limited processing capability and power, restrictive data entry methods [51], and the way the mobile context is connected to network, or other, more traditional working environments. In addition, earlier research has focused mainly on mobile consumer services and not on employee services. Thus, the characteristics of mobile work are not emphasized in existing questionnaires and models. For example, in order to be more suitable for consumer services, TAM has been extended with innovation diffusion theory, perceived risk and cost [48] as well as value, trust and ease of adoption [19]. Moreover, in order to explain customers mobile Internet adoption, Kim, Chan and Gupta have developed the Value-based Adoption Model, which is also based on TAM [25]. In this model, the perceived value of mobile Internet is determined by usefulness, enjoyment, fee and technicality. Usage fee and enjoyment are important issues in consumer settings because consumers adopt and use mobile services voluntarily for personal purposes. The work context is somewhat different as employees are using mobile business services for work purposes and the cost of mandatory adoption and usage is born by the organization. [25] As companies pay for mobile business services, they also want some return on their investments. Thus, the adoption and use of mobile business services must also have individual and organizational impacts. The linkage between technology and individual performance is addressed, for example, in the model of information system (IS) success [7], [8]. According to the IS success model, system quality, information quality and service quality affect intention to use, or use and user satisfaction. In addition, the amount of use may have a positive or negative effect on the degree of user satisfaction and vice versa. Moreover, the use and user satisfaction lead to an individual impact, and this impact should eventually have some organizational influence. Task-technology fit (TTF) is another model that emphasizes individual impact [15]. According to TTF, individual impact refers to improved efficiency, effectiveness, and/or higher quality. Good fit between task and technology is assumed to increase the likelihood of utilization and also to increase the performance impact since the technology meets the task needs more closely [15]. TTF has also been applied in mobile work contexts, for example, in police field work [17], mobile electronic procurement [13] and insurance industry [28]. Although it is well understood that mobility and mobile context have remarkable effects on the use of mobile applications (e.g., [10], [49]), these aspects are seldom included in the models presented in this Section 2. As Gebauer and Tang [14] point out, user mobility and the context in which technology is being used need to be taken into account when applying TTF to mobile information systems. The context in which the mobile work is performed is challenging because of its dynamic nature. The battery life of the device, lighting, temperature, noise, or network connections are examples of the factors that may change while performing mobile work tasks. All these factors affect the users ability to perform work tasks with a mobile service. In order to have a comprehensive view of the aspects that affect the use of mobile business services in work context, we use the concept of mobile business service experience. In this paper, this
2. BACKGROUND
According to ISO 9241 standard [18], usability is the extent to which the intended users of a product achieve specified goals in an effective, efficient and satisfactory manner within a specified context of use. The usability of the service affects the user experience (UX), which is one of the key elements of a successful mobile service [43]. UX is a wider concept than usability but it cannot be easily defined [12]. User experience includes, for example, the tasks, the service provided, navigation, the design and the value the user achieves when using the service [43]. Furthermore, Roto argued that the main elements of UX are users internal state, context of use, and the actual system [37]. In our study, we state that identifying and evaluating impacts of user experience are vital for improvement, and therefore by minimizing cases where these impacts are incorrectly estimated, we can support the successful development of mobile business services. Many generic usability questionnaires have been developed for evaluating software systems, for example, the System Usability Scale (SUS) [2], the Software Usability Measurement Inventory (SUMI) [46], the Technology Acceptance Model (TAM) [6], the Questionnaire for User Interface Satisfaction (QUIS) [4], and the Post-Study System Usability Questionnaire (PSSUQ) [29]. In addition, some questionnaires have been deployed to evaluate the usability of specific software systems like web applications (WAMMI) [26] and electronic mobile products (Mobile Phone Usability Questionnaire (MPUQ)) [39]. The limitation of these questionnaires is that most of them were developed for a desktop environment and they do not cover the challenges in the usability
54
FP
concept describes the user experience of mobile business services including perceived usability of mobile business services, perceived fit for mobile working context and perceived impacts on mobile work productivity. In this way, mobile business service experience goes beyond the user experience towards understanding how employees and companies experience mobile business services. The dimensions of mobile business service experience are discussed in detail in the next section.
6. They should use their users terminology and their navigational structure should be organized in the way in which users think (User centeredness). 7. They should be adapted to each and every users own needs and capabilities (Personalization). Zhang and Adipat [51] studied current standards, handbooks and usability studies on mobile applications and identified nine generic attributes for usability of mobile applications: learnability, efficiency, memorability, errors, user satisfaction, effectiveness, simplicity (complexity), comprehensibility (readability) and learning performance. To study perceived usability of the mobile business service among users, the items described above, usefulness, efficiency, ease of use, user satisfaction, learnability etc., form the basis for these studies. Thus, the questionnaire developed should also include statements based on these items.
FP
55
established by literature reviews and interviews with managers, developers, usability specialists and customers of mobile business services. In our interviews, there were altogether 14 participants during spring 2006. All interviews were recorded and transcribed for future analysis. The interviews were analyzed qualitatively. In the following sections, the refinement work for the questionnaire will be described more specifically.
56
FP
environment, as well as the overall system context (see, e.g., [7]). How a mobile business service suits a mobile working context is measured in our questionnaire by four dimensions: a mobile device (e.g., screen size, battery capacity, using with one hand), environmental effects (e.g., work safety), a mobile service and work tasks. All these dimensions include a view of mobility; using a mobile service while moving. In order to more clearly identify the reasons behind the problems, we separated a mobile device from a mobile service. Finally, in order to evaluate impacts on mobile work productivity, we added 21 items. The items related to efficiency, effectiveness and usefulness are especially closely related to productivity.
in Finland. The mobile business service includes mobile work email, calendar, and employee directory. The mobile business service was being piloted at the time of the study. The sample size in this pilot study was 163. Respondents were selected by the case company using systematic sampling, so that those who were really using the service were chosen for the study. There were altogether 84 responds in this study and the response rate was 51 %. The data was gathered using a web-based survey tool. Respondents were between 27 and 60 years old. The average age of the respondents was 40 years. Most of the respondents were men (73 %). The users used the service daily. Over 60 % of the respondents had not used a similar kind of service earlier. Over 70 % of those with previous experience claimed that the previous experience had been beneficial when learning to use the service. Almost every respondent (98 %) used the mobile service via a mobile phone. The third pilot test, conducted together with a company in the area of mobile service creation, also included knowledge workers located all over the world. The solution evaluated was a combination of push email, calendar, contacts, and other mobile services, integrated into one user interface. The sample size was 100 participants. The data was gathered using a web-based survey tool. The sample size was one hundred and the response rate was 34 %. The average age of the participants was 36 years (range 2647 years). The majority of the respondents were male (86 %). Of the participants, 80 % had used a similar kind of mobile service earlier and over 77 % used the particular mobile business service evaluated more than five times per day. Their average experience of using this particular mobile business service was 26 weeks. Table 1 Summary of the pilot tests Pilot test #1 (n = 48) sample size response rate Age (mean) Gender Men Women N/A 40 5 3 60 22 2 30 2 2 100 48 % 46 Pilot test #2 (n = 84) 163 51 % 40 Pilot test #3 (n = 34) 100 34 % 36
4.2.1 Pre-testing
The questionnaire was pre-tested with four people. They completed a paper questionnaire while a researcher was observing them. The test users thought aloud as they completed the questionnaire. The questionnaire was modified with the results of the pre-tests. For example, the layout of the questionnaire form became clearer when each of the dimensions was put on separate pages. In addition, the scale was added at the beginning of every part of the questionnaire and the scale was described both numerically and verbally. Some items were also rephrased to be more unambiguous.
The total number of items was between 43 and 46 in the three pilot tests (Table 2). The first and second pilot tests included otherwise the same items but one item related to productivity (quality of output) was added to the second pilot test. This item was also included in the third pilot test. The third pilot test was integrated into the realistic ongoing mobile business service development process and the participants of our study were the same end-users who evaluated the service in the realistic development process as well. Because of the nature of the service development process, the end-users installed the service into their mobile phones by themselves. Therefore, three questions related to the installation phase were added at the beginning of the questionnaire. In addition, one item related to the mobile working context was removed from the third pilot test because of its redundancy with another item. In the third pilot test, we combined
FP
57
and reformulated four items into two and added two new items related to general functions that are useful for mobile workers [50]: information processing and mobile communication. In addition to these items, we asked how often and what problems users had experienced with the mobile business service related to using mobile business service, missing or wrong information, mobile device, network connection and environmental effects. There were also some open ended questions and questions related to the background information of the respondents for statistics. Table 2 shows the degrees of internal reliability, coefficient alpha (Cronbachs ) for all dimensions and the number of items in the pilot tests. Table 2 Cronbachs for pilot tests (# of items per dimension) Pilot test #1 (n = 48) Usability Mobile working context Mobile work productivity 0. 84 (12) 0.69 (11) Pilot test #2 (n = 84) 0.68 (12) 0.58 (11) Pilot test #3 (n = 35) 0.82 (15) 0.64 (10)
11. Ease of navigation 12. Efforts made when starting to use 13. Functions are simple to use 14. Concentration 15. Recommend to others Fit for mobile working context (16-25) 16. Ease of use with a device 17. Screen size 18. Battery capacity 19. Suitability of a device for working on the move 20. Information input 21. Using a device with one hand 22. Environmental context (coldness, sunshine, darkness, noisiness, etc.) 23. Safety risks while working on the move 24. Ease of use while on the move 25. Ease of use in a hurry
Most of the values of coefficient alpha exceeded 0.80, which is very good level of alpha [9]. The alphas related to the mobile work context were lower than the others and even at an unacceptable level ( < 0.60) [9]. Deleting some items (e.g., the item related to the environmental context) would increase the alpha of working mobile context to the level of 0.6 in all the cases (pilot tests #1, #2, #3). In the case of the pilot test #2 the alpha of usability is under 0.7 but if two of the items (related to errors and concentration) are deleted the alpha increases to over 0.7.
26. Satisfaction with efficiency at work 27. Use accelerates work performance 28. Use improves work motivation 29. Use improves work satisfaction 30. Use improves fluidity of work 31. Accomplishing work task effectively 32. Use increases productivity 33. Able to perform tasks in less time than before 34. Able to complete tasks easier 35. Satisfaction with the quality of work tasks 36. Less working phases 37. Less time to go through working phases 38. Less additional traveling 39. Better access to information needed at work 40. Decision making 41. Distributing work tasks 42. Planning and coordinating work tasks 43. Gathering information 44. Sharing information 45. Information processing 46. Mobile communications
5. RESULTS
As a result of iterative item generation, item refinement and pilot tests, MoBiS-Q, a questionnaire for measuring mobile business service experience was developed. Three dimensions of the questionnaire and items related to each dimension are presented in Table 3. Table 3 Dimensions and items of MoBiS-Q Dimension Perceived usability of a mobile business service (1-15) Items 1. Easy to install and setup to the device 2. No errors in the installation process 3. Speed of installation 4. Easy to learn 5. Easy to become skilful 6. Reliability 7. Suitability for work tasks on the move 8. Errors 9. Quick enough 10. Functions are necessary
After the development of the MoBiS-Q, three researchers also discussed the relevance of each item based on the responses. Most of the items related to the perceived usability of a mobile business service were seen as relevant to all pilot tests. For example, in all
58
FP
of the three cases, parallel results were gained for statements related to learnability, suitability for mobile work tasks and necessity of the functions in mobile service. Moreover, the results showed that all users were ready to recommend the mobile service to others doing similar kind of work. However, differences were found between the responses of knowledge workers and taxi drivers. For example, the statements related to the reliability of a mobile service, amount of attention needed to use a mobile service and simplicity of use seemed to be more critical factors for taxi drivers than for knowledge workers. This is reasonable; while taxi drivers have to operate the mobile business service while on the move (driving) whereas knowledge workers can use their service, for example, while sitting on the train, bus or in meetings. There were also some differences among the knowledge workers in results from pilot tests #2 and #3. For example, responses to statements related to faultlessness and reliability were not in line in these two tests. From these results, one could come to the conclusion that one service could have had some problems in its functionality. Some of the items related to perceived fit for a mobile working context had some differences among the responses. For example, one group of knowledge workers (the pilot test #3) thought that the battery capacity of their mobile device was not sufficient for using their mobile service. Lengthy use of a mobile service via mobile phone drains the power remarkably. However, we found that the battery capacity of the mobile device is not a relevant question for users whose mobile device is an embedded system, like taxi drivers in-car systems. In addition, using a mobile device with one hand was rated differently. The taxi drivers were more satisfied with using their mobile service with one hand whereas the knowledge workers were not so satisfied. The reason for this may be that the taxi drivers device is a touch screen and the size of the screen is larger, whereas knowledge workers were mainly using their mobile phones. For the same reasons as before, mobile work safety seemed to be a more relevant item for taxi drivers than for knowledge workers. On the other hand, the data security could be a more valuable item to be added for knowledge workers. It would be beneficial to study how data security is experienced, especially how much the users are ready to suffer without decreasing the usability of a mobile service. Sometimes data security may be cumbersome from a users point of view, for example, because of passwords or duration of sessions. Therefore, it is important for service developers to find the balance between data security and usability of a mobile service. In addition to these differences, we found that some terms are suitable for some people and some are not. For example, the terms working context, mobile work context, or context of use have different meanings for knowledge workers and taxi drivers. Working context for knowledge workers means, for example, their desktop and devices. When they send e-mail in a railway station, they do not think the place is their working context. For taxi drivers the meaning of the working context is wider. They think that the environment (e.g., a car and traffic) where they move and perform their work tasks is the working context. Related to the context of use and the complexity of mobile services, it may be difficult for the users to know why there are problems. The users do not know whether the problems are
caused by the device, the service or network connections. However, this is very important for service providers to recognize if the problem is in the service or in the device. On the other hand, it is essential to take into account that the service and the device are compatible. Environmental effects, like sunshine and cold weather, may make the use of the mobile service difficult. Therefore, it is important to cater for these when designing mobile systems for users whose context of use is not alterable. For example, taxi drivers use their mobile system while driving a car. They cannot choose the context of use and minimize the negative effects of environment. On the other hand, knowledge workers do not use their mobile service only while walking or driving. They can move to another place or indoors where the interaction with a mobile service may be better. In this way, they can minimize the environmental effects. After these dimensions, there were first ten more general productivity items related to perceived impacts on mobile work productivity (Part A). One of the items asked directly if the use of a mobile business service increases work productivity. The other statements assessed factors closely related to productivity, i.e. how satisfied the respondents were with their efficiency at work, if the use of a mobile business service was experienced as accelerating their work performance or improving their work motivation or satisfaction. In addition, there were statements asking if the use of mobile business services improves fluidity of work or enables workers to accomplish work tasks efficiently, quicker, or easier. In addition, quality of the outputs the work tasks performed was also one item assessing productivity. These items were relevant in all of the pilot tests due to their general nature. In order to understand the relevance of more specific productivity statements, the respondents could answer Does not apply to my work/ the mobile service to the eleven additional productivity items (Part B). These items were named as positive impacts on mobile work productivity. Two out of eleven items seemed to be irrelevant for knowledge workers in the second and third pilot tests: There are fewer working phases to complete now and I need less time to go through different working phases. These were, however, important for taxi drivers. Items related to availability of information, decision-making and whether gathering or sharing information has improved seemed to be relevant in all of the pilot tests. In the third pilot test, items related to information processing and mobile communication were highly relevant for knowledge workers. Some differences in the responses may be due to the nature of the work tasks. For example, taxi drivers work is routine, whereas knowledge workers use their mobile service, for example, when they have down time while traveling or between meetings. Taxi drivers also perform almost all their work tasks with the mobile service whereas knowledge workers perform only some supportive tasks with their mobile service, for example, reading and responding to emails. Thus, the estimation of productivity impact may not be comparable between such a different work tasks.
FP
59
mobile business services (the three pilot tests described earlier). We found that fast, real time, continuous and close feedback is one of the keys for successful service development cycle. The benefits that a questionnaire approach can bring are related to efficiency and scale. To add value, the tools in use need to be scalable to the needs of design and development processes [38], [27]. This requires continuous readiness for analysis and synthesis. Focus on sufficiently generic questions, with the option for open feedback, can make it easier to get continuous, fast feedback. We found that this type of analyzed and summarized information is relevant, and also needed for service development. However, focus on summaries only can make the view too narrow. Details can bring the needed added value for developing the experience further. For example, most of the respondents in this study experienced that the network connection is often or quite often the reason for bad mobile business service experience. This is a known problem, but also something that no technology or service provider can solve alone. If we focus on more system details, it is also important to remember that according to the feedback received in this study, a 15 minute questionnaire can be too long for people who need to respond to questionnaires on a continuous basis. Due to the complex nature of mobile business service, a questionnaire was experienced to be more beneficial when more than only one aspect of the service is combined in the contents. As MoBiS-Q combines usability, context and productivity factors together, the interest group for the results is wider, including not only designers and developers but also managers, sales people and customers. Therefore, co-development of questionnaire content is also needed. In any case, the responsibility for leading the planning activity needs to be clear: concentration needs to be on listening to core messages of people participating in design and development activities [38], [23], [40]. Important factors for the evaluation of business services include the way of presenting the results: the simpler the presentation of the results the easier it is to communicate the findings in the organization. Therefore, graphics is a recommended format for presenting the results, and also, if there is a need for a quick and short reporting, a single average figure or grade can be calculated and used to communicate the measured experience of a service. When the findings are presented in a simple way and the questionnaire is kept more or less the same, the results are comparable with findings based on competitors services, with different versions of a service, or even with other mobile business services. Also, if the results are easy to present, they can be used in the sales process of a newly developed service.
of 0.70 (Table 2), some items need to be rephrased or deleted. The length of the questionnaire should also be considered. Due to the speed of mobile service development, analysis, synthesis, results generation and use need to be supported by the tool in use. From the industry perspective, the two most integral and demanding phases in tool development are on the one hand the development of a dynamic but exact data algorithm, and on the other hand mapping dynamic functionality with visualization. The role of visualization is to make the cognitive processes for decision-making faster. Interaction can add flexibility, or more detail in the faster decision-making process [3], [45]. In this study, the MoBiS-Q tool does not yet contain any automatic or visual functionality for analysis or synthesis. Because of this, the time used for these phases is too long from the service design and development phase point of view, and the cognitive process for making decisions based on the synthesis, continues to be slow. Also, due to relatively small response rates, no actual statistical analysis could be used here. In addition, there needs to be easy access to the tool for any of the people belonging to the network of users, and their access needs to be to real time, analyzed data. On the other hand, security needs to be taken into account as well. Keeping security in mind, integrating the knowledge from this study with some already available, open source based statistical visualization tool could be the next iterative step towards a competitive solution [36], [41].
6. CONCLUSIONS
The primary contributions of this paper were the definition of three dimensions associated with the mobile business service experience and the development of new measurement tool, MoBiS-Q, for measuring these dimensions. These dimensions include perceived usability of mobile business services, perceived fit for mobile working context and perceived impacts on mobile work productivity. When evaluating the usability and productivity issues together, the idea was to understand the critical factors in the mobile work context. As User-Centered Design approaches recommend, it is crucial that the development of a new service is a joint operation between relevant parties and departments in an organization, including user representatives, sales, marketing, product management, technology, and usability in the process. Therefore, the MoBiS-Q questionnaire preparation and analysis can be seen as an ideal tool for bringing together all those parties to work together during the development process, composing the set of questions and evaluating the results together. The study adds value by showing that more focus on the system context related details could be the way to develop the service experience further. Cross organizational cooperation agreements and activities between different service and device providers can make the services more integrated in the context, and more appealing for the user. The benefits of the cooperationcompetition dilemma can also bring value in problem solving, and support channels. For the user it is difficult to know where the actual problem lies, but if not solved, the service experience may be bad, and the productivity of an employee may be lower. The most difficult challenges (e.g., reliability of networks while on the move) cannot be solved by any service or technology provider alone.
60
FP
Due to complexity of the system solutions, and also due to many active roles in the service network, the planning phase of the evaluation process will need some more attention, and it needs to be cooperative. Responsibilities need to be clear. Other tools will be needed, too; questionnaires cannot say it all. With this study, the cross organizational planning approach was not yet in use, therefore there were some challenges in content generation. In addition to planning, analysis, synthesis and result generation will also need more support from the MoBiS-Q tool. The tool will need to contain more automatic, statistical basic methods with automatic visualization. The idea presented here would be to enable the integration of this tool with some already available, possibly open source based, statistical visualization tool. The tool needs to be easily accessible, but access control is essential. The requirements presented here are basic requirements for wider adaptation of the tool. The process measuring the final service experience needs to be faster, more flexible and more visual.
Year Update. Journal of Management Information Systems, 19, 4, 9-30. [9] DeVellis, R.F. 2003. Scale Development Theory and Applications. 2nd edition. Sage Publications. [10] Dix, A., Rodden, T., Davies, N., Trevor, J., Friday, A., and Plafreyman, K. 2000. Exploiting Space and Location as a Design Framework for Interactive Mobile Systems. ACM Transactions on Computer-Human Interaction, 7, 3, 285321. [11] Evans, N.D. 2002. Business Agility Strategies for Gaining Competitive Advantage through Mobile Business Solutions. Upper Saddle River (N.J.): Prentice Hall. [12] Forlizzi J., and Battarbee K. 2004. Aesthetics, ephemerality and experience: Understanding experience in interactive systems. In proceedings of Designing interactive systems: processes, practices, methods, and techniques, ACM Press, 261-268. [13] Gebauer, J., and Shaw, M.J. 2004. Success Factors and Impacts of Mobile Business Applications: Results from a Mobile e-Procurement Study. International Journal of Electronic Commerce, 8, 3, 19-41. [14] Gebauer, J., and Tang, Y. 2007. Applying the Theory of Task-Technology Fit to Mobile Information Systems: The Role of User Mobility. Proceedings of the International Conference on Mobile Business. [15] Goodhue, D. L., and Thompson, R. L. 1995. Tasktechnology fit and individual performance. MIS Quarterly, 19, 213-236. [16] Gorlenko, L. and Merrick, R. 2003. No wires attached: Usability challenges in the connected mobile world, IBM Systems Journal archive, 42, 4, 639-651 [17] Ioimo, R.E. and Aronson, J.E. 2004. Police field mobile computing: Applying the theory of task-technology fit. Police Quarterly, 7, 4, 403-428. [18] ISO 9241-11. Ergonomic requirements for office work with visual display terminals part 11: guidance on usability (ISO/IEC 9421-11: 1998). [19] Kaasinen, E. 2005. User Acceptance of Mobile Services Value, Ease of Use, Trust and Ease of Adoption. VTT Information Technology, Helsinki. [20] Kakihara, M., Sorensen, C. 2004. Practicing Mobile Professional Work. Tales of Locational, Operational and Interactional Mobility. INOFO: The Journal of Policy, Regulation and Strategy for Telecommuniacation, Information and Media, 6, 3, 180-187. [21] Kalakota, R., and Robinson, M. 2001. M-Business The Race to Mobility. McGraw-Hill. [22] Karahanna, E., Agarwal, R., and Angst C. 2006. Reconceptualizing Compatibility Beliefs in Technology Acceptance. MIS Quarterly, 30, 4, 781-804. [23] Kautonen, M, Schienstock, G. and Tiainen, M. 1999. Knowledge-Intensive Business Services Their Role and Development in the Tampere Urban region. In: Schienstock, Gerd and Kuusi, Osmo (eds.): Transformation towards a Learning Economy. The Challenge for the Finnish
7. ACKNOWLEDGMENTS
This study was conducted as part of the MOMENTO project (Ensuring the success of mobile services for business usage by multidisciplinary measurement tools). MOMENTO is a research and industrial cooperation project that started at the beginning of 2006. The project is financed by Tekes (the Finnish Funding Agency for Technology and Innovation) and five companies. We would like to express our gratitude to the representatives of the case companies and their customers who participated in this study.
8. REFERENCES
[1] Barnard, L., Yi, J., Jacko, J., and Sears, A. 2007. Capturing the effects of context on human performance in mobile computing systems. Personal and Ubiquitous Computing, 11, 2, 81-96. [2] Brooke, J. SUS A quick and dirty usability scale. Available online at http://www.usability.serco.com/trump/documents/Suschapt.d oc. [3] Card, Stuart K., MacKinlay, J. and Schneiderman, B. 1999. Information Visualization: Using vision to think. Academic press. [4] Chin, J.P., Diehl, V.A., and Norman, K.L. 1988. Development of an instrument measuring user satisfaction of the human computer interface. Proceedings of CHI88, 213218. [5] Corso, M., Giacobbe, A., Martini, A., and Pellegrini, L. 2006. What Knowledge Management for Mobile Workers? Knowledge and Process Management, 13, 3, 206-217. [6] Davis, F.D. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, September, 318-340. [7] Delone, W.H., and McLean, E. R. 1992. Information systems success: The quest for the dependent variable. Information Systems Research, 3, 6095. [8] DeLone, W.H., and McLean, E.R. 2003. The DeLone and McLean Model of Information Systems Success: A Ten-
FP
61
Innovation System. Finnish National Fund for Research and Development. Helsinki. [24] Kemppil, S. and Lnnqvist, A. 2003. Subjective Productivity Measurement. The Journal of American Academy of Business, 2, 2, 531-537. [25] Kim, H-W., Chan, H.C., and Gupta, S. 2007. Value-based Adoption of Mobile Internet: An empirical investigation. Decision Support Systems, 43, 111-126. [26] Kirakowski, J., & Cierlik, B. 1998. Measuring the usability of web sites. In Proceeding of Human Factors and Ergonomics Society 42nd Annual Meeting, Santa Monica, CA. [27] Larman, G. 2004. Agile & iterative development - A manager's guide. Addison Wesley. [28] Lee, C-C., Cheng, H.K., and Cheng, H-H. 2007. An empirical study of mobile commerce in insurance industry: Task-technology fit and individual differences. Decision Support Systems, 43, 95-110. [29] Lewis, J.R. 1995. IBM Computer usability satisfaction questionnaires: psychometric evaluation and instructions of use. International Journal of Human-Computer Interaction, 7, 1, 57-78. [30] Markova, M., Aula, A., Kulju, M., Vainio, T. and Wigelius, H. 2007. MoBiS-Q - A tool for evaluating the success of mobile business services. In Proceedings of MobileHCI 2007, 238-245. [31] Markova, M., Aula, A., Lnnqvist, A. and Wigelius, H. 2008. Identifying and measuring the success factors of mobile business services, International Journal of Knowledge Management Studies, 2, 1, 4963. [32] Nielsen, J. 1993. Usability engineering. Academic Press Inc.,Boston. [33] Oinas-Kukkonen, H. and Kurkela, V. 2003. Developing Successful Mobile Applications. International Conference on Computer Science and Technology (IASTED), 50-54. [34] Perry, M., OHara, K., Sellen, A., Brown, B., & Harper, R. 2001. Dealing with Mobility: Understanding Access Anytime, Anywhere. ACM Transactions on ComputerHuman Interaction, 8, 4, 323-347. [35] Rangone, A., and Renga, F. 2006. B2e mobile internet: an exploratory study of Italian applications. Business Process Management Journal, 12, 3, 330-343. [36] Rhyne, T.-M. 2005. Visualization viewpoints. http://ieeexplore.ieee.org/iel5/38/35448/01683688.pdf?tp=&i snumber=&arnumber=1683688. [37] Roto, V. 2006. Web Browsing on Mobile Phones Characteristics of User Experience. Doctoral dissertation,
TKK Dissertations 49, Helsinki University of Technology, Finland. [38] Rosenbauhm S., Rohn J. A. and Humburg J. 2000. A Toolkit for Strategic Usability: Results from Workshops, Panels & Surveys. CHI 2000 Proceedings. ACM, Inc. [39] Ryu, Y.S., and Smith-Jackon, T.L. 2006. Reliability and Validity of the Mobile Phone Usability Questionnaire (MPUQ). Journal of Usability Studies, 2, 1, 39-53. [40] Saint-Onge, H. and Wallace, D. 2003. Communities of Practice for Strategic Butterworth & Heineman. Leveraging Advantage.
[41] Sasse, M.A., Flechais I. 2005. Usable security. In: Cranor F. and Garfinkel S. Designing secure systems that people can use. OReilly. [42] Savio, N. and Braiterman, J. 2007. Design Sketch: The Context of Mobile Interaction. In Proceedings of MobileHCI 2007, 284-286. [43] Schonberg, E., Cofino, T., Hoch, R., Podlaseck, M., and Spraragen. N, S. L., 2000, Measuring success. Commun. ACM, 43, 53-57. [44] Sheng, H., Nah, F.F., and Siau, K. 2005. Strategic implications of mobile technology: A case study using Value-Focused Thinking. Journal of Strategic Information Systems, 14, 269-290. [45] Siirtola H. and Mkinen, E. 2005. Constructing and reconstructing the reorderable matrix. Information Visualization 4, 1, 32-48. [46] Software Usability Measurement Inventory. Information on line at http://www.ucc.ie/hfrg/questionnaires/sumi/. [47] Verburg, R.M., Testa, S., Hyrkknen, U. and Johansson, N. 2006. Case descriptions of mobile virtual work in practice. In Andriessen, J.H.E. and Vartiainen, M. (Eds.) Mobile virtual work, Springer, Heidelberg, 267-288. [48] Wu, J-H., Wang, S-C. 2005. What drives mobile commerce? An empirical evaluation of the revised technology acceptance model. Information & Management, 42, 719-729. [49] York, J., Pendharkar, P.C. 2004. Human-computer interaction issues for mobile computing in a variable work context. International Journal of Human-Computer Studies, 60, 5-6, 771-797. [50] Yuan, Y., Zheng, W. 2006. The fit between Mobile Task and Mobile Work Support: A Theoretical Framework. Proceedings of the International Conference on Mobile Business. [51] Zhang, D., Adipat, B. 2005. Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications. International Journal of Human-Computer Interaction, 18, 3, 293-308.
62
FP