Email: ANMIMA@ECON.JYU.FI

The purpose of this paper is to demonstrate the relationship and dependencies between different dimensions of the seamless use experience in an electronic environment. This paper outlines the factors affecting the seamless use experience in both mobile and fixed-line Internet services from the customer's perspective. The customer's perception about seamless use experience results mainly of the experienced interaction in the customer-technology interface. However, our results show that the seamless use experience is defined by many more variables such as satisfaction towards the service provider and customer's previous use experience. The results are based on a large consumer survey conducted among mobile and fixed-line Internet users in Finland during summer 2003.   

Keywords: seamless, electronic channels, customer interface

Several extensive collections of general user interface guidelines (Brown 1988, Marshall et al. 1987, Mayhew 1992, Smith and Mosier 1986) and methodologies (e.g. LUCID, Kreitzberg 1996) to develop and to enhance user interfaces has been published. In our study we are not trying to define the place of a key on a cell phone or the place of an icon on a screen, but instead, the focus is on the seamless use of the mobile and fixed-line services itself. Designers' assumptions shape their predictions about the final product and how it might perform in interaction with its users (Hasdogan 1996). However, the predictions designers make about the product's usage and performance do not always match the expectations of the user and actual usage of the product (Nogier 2001). Such mismatches between the designers' predictions and actual product use are likely to lead to customer dissatisfaction and frustration with the product.

Traditionally usability testing has been device and technology specific. The main usability research method has been non-participatory observation in laboratory settings with a very small sample size (Ivory et al. 2001). The satisfaction has been measured in relation to the system (Nielsen 1993, 33). However, experiments have shown that usability of a service cannot be predicted from the technical quality of its components. This is to a large extent due to the interaction between objective performance measures and functionality (Boves et al. 1999). Usability becomes seamless use experience, when the service use context and content are taken into account. Seamless use experience research is more customer-centric analyzing the value proposition of a service throughout the value chain. A common research method in seamless use experience studies is survey (Ivory et al. 2001) and the customer satisfaction is often measured in relation to the service and content provider.

In this paper we focus on five usability attributes previously defined by Nielsen (1993, 26-37): learnability, efficiency of use, memorability, satisfaction and errors. Nielsen has concentrated in his research on the graphical user interface usability testing. However, these attributes can be easily understood also when put into a context such as mobile or fixed-line services. Nielsen's attributes represent the most generic model among the usability research. The purpose of this study is the map the different dimensions of seamless use experience in the electronic service context. Therefore, Nielsen's model provides us a suitable starting point.  

Other definitions do exist including more or less overlapping attributes. Different definitions can include the same attributes, which are defined dissimilarly. ISO 9241 Part 11 (1995) introduces definition of usability of a given computer system and uses three attributes: efficiency, effectiveness and satisfaction, in which efficiency includes learnability.  Dix et al. (1997, 162-175) defines three main categories: learnability, flexibility and robustness. They are further defined to include more specific principles supporting the main categories. For example, Dix et al. (1997, 163) define learnability as including principles like predictability, synthesizability, familiarity, generalizability and consistency. For mobile systems, collection of usability properties would include intuitiveness, ease of use, efficiency of use and reliability (Clevenger 2002).

With learnability as a usability attribute we refer to the difficulties customers experience when trying to learn how to use the electronic service. At least novice users tend to think that mobile services are expensive to use (Munnukka et al. 2003), so it is important that services are easy to use right from the beginning. These novice users might not want to spend their money on learning. Highly learnable systems have steep incline in the beginning of the learning curve, which means that it is relatively easy for the users to learn to use the system within short time period).  The most common way to measure proficiency is to check whether the users are able to complete particular tasks successfully.

Efficiency of use has two aspects, effectiveness and efficiency (e.g. ISO 9241 Part 11 1995). Efficiency of use in usability context normally refers to expert user's normal performance level (Nielsen 1993, 30). In certain systems average users might not even reach that performance level. In our study context efficiency of use refers to effectiveness and efficiency. By effectiveness we mean how accurately and precisely customers achieve their goals of usage. By efficiency we refer to the customers' spent resources including time and money.

Electronic services should be easy to remember, which contributes to the memorability attribute. This includes finding the service and being able to easily access the service repeatedly. The customers shouldn't have to learn to use the service again and again. Instead, the service should be designed in a way that the customers feel comfortable as they return to use it even though the use of the service/services might be non-regular.

Bowen and Chen (2001) found that the relationship between customer loyalty and satisfaction is non-linear. According to Coyne (1989) there are two critical thresholds affecting the link between satisfaction and customer loyalty. On the other side, when customer satisfaction reaches a certain level, the loyalty increases very strongly, and at the low end, when satisfaction decreases to a certain point, the loyalty drops very strongly (Oliva et al. 1992). Using the electronic service should constitute a pleasant experience leaving the customer with a feel of satisfaction and making her long for re-using the service.

Errors as a usability attribute in our context is two-fold like efficiency of use. We refer to two kinds of errors, namely minor and catastrophic errors (Nielsen 1993, 31). Minor errors hinder the use of the electronic services, but don't affect the final outcome. Minor errors include typos, using wrong links, pressing wrong keys and so on. Minor errors have been found to be closely interrelated with efficiency of use (for example Nielsen 1993, 32). Catastrophic errors lead into a situation, in which the customer is unable to finish the use of electronic service in a desired way. The customer may be left without a confirmation of a successful transaction, the use session cuts off and so on. Minor errors are such that customers should be able to recover easily from them whereas catastrophic errors have far reaching effects and should never happen. 

In our research we are trying to see usability in a larger scale, and therefore we use the term seamless use experience, which refers to the service as a whole. What is hindering the usage of the services, what do customers really want to accomplish with the service, what are the ultimate goals of the customers, are they really satisfied with the service level and so on. We are trying to provide the service developers tools to create more focused services by knowing which features in the service should be stressed and in which context. The customers waste their own time and money on these services, and if they are not satisfied they are more likely to change the service provider.

WAP-based services, especially when accessed via very small screens, have suffered a great deal of criticism from both the popular press and technology experts (e.g. Nielsen 2000, Weeks 2000). Contrary opinions have been also presented. According to Buchanan et al. (2001), most people find the basic WAP scheme easy to learn and simple to interact with. However, the WAP system was found ineffective to complete tasks and error-prone. One must keep in mind though, that Buchanan et al. (2001) conducted their survey among 110 students who were also experienced mobile phone users.

In the case of Web site design, too much attention has been paid to the aesthetics, which lead to amazing looking Web sites but actually cause frustration because customers have difficulty in finding what they are looking for. A Web site should reflect the value proposition which satisfies the customer needs to ensure repeat visits (Cox et al. 2002). Cox et al. (2002) categorize six key Web site quality factors affecting the seamless use experience: clarity of purpose, design, accessibility and speed, content, customer service and customer relationships. The above mentioned factors reflect the usability of the Web site during customer navigation and aim to reduce customer frustration. It has been found that the design of a Web site will affect customers' decisions to include online shopping as a channel option (Schoenbachler 2002). As the marketer takes a more customer-centric approach and focuses on the consumer rather than the channel, many of the multi-channel challenges fade away.

New mobile devices and services have been found to be more realistic and useful than the previous models. They give customers a seamless service even when customers are moving between network connections (Nielsen 2003). Seamless use experience reduces sales costs and shortens sales cycles, which improves Return On Investment (ROI) (Rhodes 2001). Products which are easier to use are easier to sell. Besides matching services and delivery channels, the attributes of the service delivery channels themselves will also be influential when customers make channel choices (Black et al. 2002). Whether or not a customer has access to a channel (e.g. does the consumer have a mobile Internet access?) is probably the most basic determinant of the set of channels considered. After solving this issue, factors such as convenience, costs and risk need to be addressed, because seamless use experience affects the service delivery channel choice. For example, usability refers to the same level of technical system quality regardless of the use location. Accessing Internet at home or in a public library represents the same level of experienced usability. But accessing Internet at home is more convenient and therefore more likely to score higher on the scale of seamless use experience than getting to a public library in a snowstorm, queuing there to get access to the computer, and so forth.

In this study, the mobile Internet is defined as usage of Internet via handheld devices such as mobile phones of PDAs. The aforementioned attributes by Nielsen (Nielsen 1993, 26-37) were chosen as they are rather generally laid out. For example, the division presented by Dix et al. (1997), refers more towards a development of user interfaces alone. It wasn't in our research interests to make such a granulated division beforehand but to investigate first and base the interpretations about the dimension division on the data collected. Before the actual data collection, focus group interviews among expert users were conducted. The meaning of these interviews was to map the possible options for survey questions. The questionnaire was pre-tested on a group of 60 students and modified accordingly.

A postal survey was conducted in May 2003. The sample was drawn from TeliaSonera (TeliaSonera is the largest mobile operator in Sweden and Finland, the second largest operator in Norway, and the fourth largest operator in Denmark) Finland's customer database.  The sample was stratified in three active user segments of mobile users, fixed-line users and combined users equal in size depending on the main electronic service delivery channel in their use. The questionnaires were tailored respectively. We call the customers, who did not own according to the database a private fixed-line connection at home, the Mobile users. The customers collected under this sample had the highest volume of mobile data transfers (GPRS, high-speed data) during the last six months in comparison to other customers in the database. They represented in every way the most active mobile Internet users the database had. The Combined users had a private fixed-line Internet connection in use at home. Further, their customer record showed active usage of mobile Internet (GPRS, high-speed data) connection and WAP-services during the last six months. The Fixed-line users owned a mobile phone and they were using regular mobile phone services such as SMS. There was no sign of Internet related activities during the last six months in their customer record. They had a private fixed-line Internet connection (mainly ADSL) in use at home.

After a second follow-up, 778 responses were accepted under further analyses. The final response rate was 25.9%, which is acceptable according to economic science standards. The response rate is normal considering the research method (postal survey) and the profile of the sample (mobile or Internet customers). In the future, it may be beneficial to consider alternative survey methods such as online or SMS-based surveys. A small minority of respondents reported using mobile phone (GRPS or high-speed data connection) as a modem in connection with a laptop as their primary electronic delivery channel. Among the Mobile users there were 16 such a customers and among both the Fixed-line users and the Combined users two in each segment.

The respondents were asked to fill out a structured questionnaire on a 7-point Likert scale concerning their preferences, experiences and beliefs towards usage of mobile and Internet services. There were up to 27 questions in each tailored questionnaire. The Mobile users were answering mobile Internet specific questions whereas the Fixed-line users were answering fixed-line Internet specific questions. As the Combined users segment had knowledge on both types of electronic services and delivery channels, half of them received a questionnaire regarding the mobile Internet seamless use experience and the other half was answering to questions concerning the fixed-line services. The survey questionnaire included questions concerning the respondent's basic demographic variables, psychological determinants such as level of innovativeness and mobile Internet usage, which was further categorized under for main themes: usage context, service content, seamless interface dimensions and use experience.

Besides trying to establish causality between the different dimensions of seamless use experience, we used somewhat ethnographic approach in our survey. Literature (Cooper et al. 1995; Järvenpää et al. 1997; Crisp et al. 1997) as well as prior conducted surveys guided us in defining the scales to measure the customers' perceived seamless use experience. To get a more accurate and objective results, the mean value of the respondents' subjective responses were calculated and used as the basis of our evaluation. Statistical methods such as ANOVA, crosstabulation, correlation coefficients, rotated factor analyses, Chi squares and finally structural modeling (AMOS) were applied to our data. Only results relevant to this paper are presented in here. Cronbach's alpha was used to measure the reliability of the results.

The demographic profile of the respondents is presented in table 1. One third (33.9%) of the respondents were women and two thirds (64.8%) were men. The majority (59.8%) of the respondents were 25-49 years old and their annual household income (28.1%) before taxes was in a range of 20 000 – 30 000 euros, which matches with the average annual income of two adults family in Finland. Only every fifth (18.2%) of the respondents had two or more children living at home. The majority of all the respondents were workers (40.6%). This result is compatible with the result of the educational background of the respondents, which was in most cases (29.0%) vocational school. Obviously Internet and its services are becoming available for all the consumer segments regardless of their annual household income or educational background.

Table 1. Profile of respondents

In overall, fixed-line connection seems to remain the most popular means of accessing Internet among all user segments. Even those respondents, who according to the operator's database had the highest amount of mobile Internet data transfers on their personal accounts compared to other customers, reported using fixed-line Internet services daily and more often than mobile Internet services (see table 2). Among the Finnish population in general, fixed-line Internet connection is very common and over 90 percent have it on use. Mobile phone (GPRS or high speed data transfer) as a modem in connection with a computer is used only monthly by all user segments. All segments are using personal services mainly occasionally and self-services tend to be used more frequently compared to personal services.

Table 2. Internet usage

The most popular mobile Internet service (α=0.8321) appears to be ordering occasionally ringing tones (50.8%) and logos (50.1). The most popular weekly used mobile Internet services were SMS-chat (6.8%) and using Multimedia Messaging Services MMS (3.7%). Fixed-line Internet connection (α=0.7206) was always (24.8%) or often (41.9%) used for banking. It was also often used for information search (42.7%) and communication (37.6%).

The respondents were asked to join different dimensions of seamless use experience with different Internet service delivery channels either according to their perceptions or actual use experience. Both segments of Mobile and Fixed-line users reported similar kind of beliefs about the dimensions of seamless use experience in different electronic channels. All the dimensions were rated more negatively in the case of mobile Internet than in the case of fixed-line Internet. Mobile Internet was seen as prone to errors (Mobile users 29.5% and Fixed-line users 13.5%) and difficult to remember (Mobile users 33.2% and Fixed-line users 21.3%) whereas the fixed-line Internet was seen easy to learn (Mobile users 75.6% and Fixed-line users 81.3%) and seamless to use (Mobile users 75.0% and Fixed-line users 76.2%).

The Fixed-line users trusted their primary delivery channel (fixed-line connection) even when feeling busy (56.6%) or traveling (18.0%). They also perceived mobile phone as a modem in connection with a laptop as free from time and place (21.9%) as the mobile Internet connection via mobile device (23.2%). It can be concluded that when a fixed-line Internet connection is not available, a typical Fixed-line user would be likely to choose the mobile phone as a modem the next best connection option. The Mobile users felt that mobile Internet is the most independent from time and place (66.0%) and also the easiest to use when traveling (54.0%). The Mobile users didn't seem to believe in the mass adoption of mobile Internet services' usage as they estimated that only 4.6 percent among their peers is currently using mobile Internet services.

According to the conducted crosstabulation, the more customer is using a certain delivery channel, the higher the perceived seamless use experience towards that channel. Respondents, who are daily using fixed-line Internet connection, also feel that it's very seamless way of accessing Internet-based services. Similarly among the Mobile users, the perception of seamless use experience grows in relation to their mobile Internet usage frequency. Correspondingly the perception of seamless use experience towards fixed-line Internet lowers among the Mobile users as their daily mobile Internet usage grows and fixed-line Internet usage decreases.

An exploratory factor analysis was used in order to identify underlying constructs and investigate relationships among key survey interval-scaled questions regarding seamless use experience (see tables 3-5). It was necessary to repeat the factor analyses among each user segment to capture to possible differences in how they define seamless use experience. Principal Axis Factoring was carried out, followed by Varimax Rotation with Kaiser Normalization. The Kaiser-Meyer-Olkin (KMO) measures of sampling adequacy (0.91-0.94) were well above the 0.5 recommendation level, and Bartlett's test of sphericity (p=0.0 and p=0.0) provided as well support for the validity of the factor analysis of the data set (Malhorta 1999). Varimax Rotation facilitated interpretability. In addition, Gronbach's alphas were counted for each factor scoring above the level of acceptance set by Nunnally (1978). Hence, the data set can be defined as reliable.

Initial runs based on a screen plot and eigenvalues showed support for five factors. Only factors with eigenvalues above 1 were expected. In our analyses the eigenvalues varied between 5.6475 and 2.4532. The criterion for assignment of reasons to a certain factor was a minimum factor loading of 0.5. The factors were labeled as learnability, satisfaction, errors, memorability and efficiency of use respectively.

Examination of the factor analysis for the dimensions of seamless use experience (presented in tables 3-5) suggests that the efficiency of use and memorability, and learnability and memorability are interrelated. The better one is able to learn the use of service, the easier its use is to remember and the more efficiently it can be exploited. The common factors for all user segments are bolded, the distinct factors are on italic and the common ones but loading a different factor are underlined.

Table 3. Seamless use experience in the case of mobile Internet

Table 4. Seamless use experience in the case of mobile Internet

Table 5. Seamless use experience in the case of Fixed-line Internet

Factors one, two and four appear to be defined by a mix of items that are mutual to all user segments (see bolded in tables 3-5). For learnability the common dimensions segment wise are previous experience in the use of electronic services and previous experience in the user of technical devices. For satisfaction we found four common dimensions: problems are treated with discretion and confidentially, operator offers after-sales services suitable for customer's needs, operator provides secure data transfers and operator is never too busy to answer customer's questions. For memorability the common dimensions among different user segments were service functions in use resemble each other and device features remain constant regardless of the product generation.  Vice versa, some of the dimensions were descriptive only for one user segment (tables 3-5 on italic). Furthermore, some of the dimensions accounted for the total variance of various factors depending on the user segment (table 6, also tables 3-5 underlined).

Table 6. Descriptive elements of seamless use experience by user segments

In the case of mobile services, the interrelated placement of keys is part of efficiency of use whereas in the case of fixed-line electronic services it's part of memorability. Written manuals affect the memorability of fixed-line electronic services and the learnability of mobile services. It can be concluded that customers perceive the usage of fixed-line Internet easier to learn than mobile Internet. It follows, that written instructions are rather needed when trying to remember how to use the services than when trying to learn how to use the service. In the case of mobile Internet, support from the written instructions is sought already during the learning period. Structure and logic of navigation constitutes learnability for seamless mobile service usage and memorability for seamless fixed-line electronic service. The constancy of service quality constitutes to efficiency of use for seamless mobile service usage and memorability for seamless fixed-line electronic service. Amount of unnecessary information within the service has a similar kind of effect on seamless use experience as the above mentioned service quality. When service is unworkable, the Mobile users perceive it as an inefficiency of use and the Fixed-line users relate it to errors in use.

On a scale of -3 to 3, efficiency of use and errors are seen as the most important usability attributes for both mobile and fixed-line Internet as an electronic service delivery channel. The mean value of efficiency of use for mobile services is 1.98 (s.d. 1.362) and for fixed-line electronic services 2.17 (s.d. 2.242). The mean value of errors for use of mobile services is 2.02 (s.d. 1.052) and fixed-line electronic services 2.13 (s.d. 1.017). However, even if both attributes are seen almost equally important, the performance of efficiency of use seems to get higher ratings among respondents (mobile -0.17, s.d. 1.558; for fixed-line 1.43, s.d. 2.330). This implies that the efficiency of use is taken better under consideration while designing the fixed-line electronic services or the implementation has been more successful in the eyes of the customers.

The importance of memorability attribute is perceived as the least meaningful by each segment. Perhaps customers don't feel there are many things to remember in the usage of electronic services. Also, if customers are using an average of three mobile services as the results of this study indicate, usage of such a small number of frequently used services can be easily memorized. The performance of memorability was rated low for both channels (mobile -0.25, s.d. 1.406; fixed-line 0.64, s.d. 1.830) but doesn't necessarily lead to extensive actions by the marketers and designers as the importance of this factor was also rated low (mobile 1.37, s.d 1.179; fixed-line 1.28, s.d. 1.463).

The seamless use experience dimension of satisfaction appears to be service provider related rather than service content or device specific. For example, the promised speed of data transfer by the operator (r = .271, p<0.01) and the installation of Internet connection in time (r = 0.160, p<0.001) correlate with the satisfaction. Different dimensions may be mutually descriptive for several factors whereas one dimension may be descriptive only for one particular factor. We have a reason to believe that the different dimensions of seamless use experience vary also depending on customer's demographic, technographic and psychographic profile. By knowing customer and service delivery specific dimensions of seamless use experience, marketers are able to focus on accurate dimensions describing each factor. When one may be lacking learnability, other's memorability may be needing attention.

Black, N. & Lockett, A. & Ennew, C. & Winklhofer, H. & McKechnie, S. (2002). "Modelling consumer choice of distribution channels: an illustration from financial services", International Journal of Bank Marketing, Vol. 20, No. 4.

Boves, L. & Os, E. (1999). "Applications of speech technology: designing for usability", The 1999 International Workshop on Automatic Speech Recognition and Understanding, Keystone, Colorado.

Bowen, J. & Chen, S-L. (2001). "The Relationship between Customer Loyalty and Customer Satisfaction", International Journal of Contemporary Hospitality Management 15/5.

Brown, C. (1988). "Human-Computer Interface Design Guidelines". Ablex, Norwood.

Buchanan, G. & Farrant, S. & Jones, M. & Thimbleby, H. & Marsden, G. & Pazzani, M.(2001). "Improving Mobile Internet Usability", in Proceedings of WWW10, Hong Kong.

Clevenger, N. (2002). "Usability within the Mobile Paradigm", available at www.pocketpcmag.com/Jul02/e_paradigm.asp as on 6.3.2003.

Cooper, D. & Emory, C. (1995). "Business Research Methods", 5^th edition, Richard D. Irwin Inc., Chicago.

Cox, J. & Dale, B. (2002). "Key quality factors in Web site design and use: an examination", International Journal of Quality & Reliability Management, Vol. 19, No. 7.

Coyne, K. (1989). Beyond Service Fads – Meaningful Strategies for the Real World, Sloan Management Review, Vol. 30.

Dix, A., Finlay, J., Abowd, G., Beale, R. (1997). "Human-Computer Interaction", 2nd ed., Prentice Hall, Glasgow.

ISO 9241 Part 11 (1995). "Ergonomic Requirements for Office Qork qith Visual Display Terminals (VDT's) – Part 11: Guidance on usability", International Organization of Standardization.

Ivory, M. & Hearst, M. (2001). "The State of the Art in Automating Usability Evaluation of User Interfaces", ACM Computing Surveys, Vol. 33, No. 4.

Kreitzberg, C. (1996). "Managing for Usability", I Alber, Antone, F. (Ed.), Multimedia: A Management Perspective, Wadsworth, Elmont.

Malhorta, N. (1999). "Marketing Research: An Applied Orientation, 3^rd ed.", Prentice-Hall, Sydney.

Marshall, D., Nelson, C., Gardiner, M. M. (1987). "Design Guidelines", In Gardiner, M. M., Christie, B. (Eds.), Applying Cognitive Psychology to User-Interface Design. John Wiley & Sons, Chichester.

Mayhew, D. J. (1992). "Principles and Guidelines in Software User Interface Design", Prentice Hall, Englewood Cliffs.

Munnukka, J., Mattila, M. and Härkönen, A. (2003). "Bundle Pricing in Mobile Services Business", in proceedings of the 2nd International Conference of Mobile Business, Vienna, Austria.

Nielsen, J. (1993). "Usability Engineering", AP Professional, New York.

Nielsen, J. (2000). "WAP backlash", available at www.useit.com/alertboc/20000709.html as on 9.7.2000.

Nielsen, J. (2003). "Mobile Devices Will Soon Be Useful", available at www.uset.com/alertbox/20010916.html as on 23.1.2003.

Nogier, J-F. (2001). "De l'ergonomie du logiciel au design des sites web", in the Proceedings of the 6th Conference on Human Factors & the Web, Austin, Texas.

Nunnally, J. (1978). "Psychometric Theory, 2nd ed.", McGraw-Hill, New York.

Oliva, T. A., Oliver, R. L. and MacMillan, I. C. (1992). "A Catastrophe Model for Developing Service Satisfaction Strategies", Journal of Marketing, Vol. 56.

Rhodes, J. (2001). "A Business Case for Usability", available at www.webword.com/moving/businesscase.html as on 2.4.2003.

Schoenbachler, D. & Gordon, G. (2002). "Multi-channel shopping: understanding what drives channel choice", Journal of Consumer Marketing, Vol. 19, No. 1.

Smith, S. L., and Mosier, J. N. (1986). "Design Guidelines for Designing User Interface Software". Technical Report MTR-10090, The MITRE Corporation, Boston.

Weeks, R. (2000). "Developers notebook: is WAP dead? Who cares?", available at www.anywhereyougo.com as on 4.2.2003.