User-Centred Product Creation in Interactive Electronic Publishing

User needs for mobile electronic information services

INFORM Workshop, 20 March 2003

It is expected that increasing number of users will want to access electronic information services with various devices and platforms, via different networks, at any time and from any place, for leisure and entertainment, and for business purposes. Do developers of innovative electronic information services take user needs into account appropriately to achieve success?

The aim of this workshop was to establish a view on the state-of-the-art of user needs analysis for mobile electronic information services in electronic publishing projects. Members from developments teams were given the opportunity to exchange their knowledge about and their experiences with user needs analysis and to discuss specific questions related to this topic. The workshop was designed to increase the awareness for useful techniques and methods for user needs analysis.

The event was organised by ACit GmbH (www.acit.net) and hosted by INTRACOM (www.intracom.gr) in association with the EC INFORM project (elpub website, www.elpub.org) which is funded by the Interactive Electronic Publishing sector of the IST programme.

The twenty-nine participants of the workshop, mostly from Greek industry and Greek universities, are practitioners involved in EU funded projects which develop mobile electronic information services. Ten participants had registered but were not able to attend the workshop, partly due to problemtic traffic conditions on that day. Some sent excuses and repeated their interest in the workshop topic and in the presentations and conclusions of the workshop (which we publish in the form of this summary together with the powerpoint presentations).

Sofia Tsekeridou from INTRACOM welcomed the participants. Ian Johnson, the coordinator of the INFORM project introduced to the objectives of the INFORM project and the elpub website. Albert Gauthier from DG Information Society of the European Commission reported about the VNET5 project which had supported electronic publishing products for a period of two years with hands-on workshops, coaching and study material (pdf, 1153 KB). Elke-Maria Melchior, the organisor of the workshop and former coordinator of the VNET5 project invited the workshop participants to join the VNET5 network (www.vnet5.org). The idea for the workshop theme emerged while the VNET5 team was coaching electronic publishing projects in IST, several of those developing mobile electronic information services. The majority of questions to the VNET5 team were related to the elicication and analysis of users needs and methods and techniques for adequate user needs analysis.

First session: Thematic overview

Tom Bösser from Kea-pro GmbH in Switzerland described three levels of maturity for user-centred product creation (in analogy to the capability maturity model applied in software engineering) and explained how competence and maturity for user-centred product creation can be increased. He considers learning from experienced experts (by coaching) as the most effective way to develop competence for user oriented activities and to increase the quality of user oriented activities in a development project. A rule of thumb is that three typical projects should be carried out with competent supervision to achieve a level of professional competence (pdf, 247 KB).

Elke-Maria Melchior from ACit GmbH in Germany introduced to user needs analysis and compared it with the related approaches (traditional systems analysis, market analysis, benchmarking). User needs are needed for system specification, user interface design and to determine criteria for user validation. Procedures and relevant techniques for the analysis of user characteristics, of user goals and tasks, and of the environment in which a new application will be used were described. Especially developers of consumer services find it difficult to analyse user needs. Consumers often lack an understanding of innovative services and find it difficult to predict and express their needs. To avoid the risk of failure user needs should be elicited and verified with design solutions and prototypes as early as possible in the development process (pdf, 1184 KB).

Anne J.C.E. Jansen from Noldus Information Technology b.v. in the Netherlands demonstrated professional software and instrumentation for behavioral research. She gave an overview of tools for the identification of consumer preferences and for user testing. She demonstrated the use of the Observer tool for the assessment of effectiveness, efficiency and user satisfaction. She showed a new, wireless camera for user testing of mobile devices and applications in the field. Noldus offers tailor-made solutions for usability labs, portable usability labs, and devices for mobile data collection (pdf, 839 KB).

Second session: Practitioners report from IST projects on their analyses of needs and requirements of users

The speakers briefly described the objectives of their project and then explained how user needs were elicited and analysed, which techniques worked well and where shortcomings were detected.

Kostas Chorianopoulos from Athens University of Economics and Business (AUEB) reported the lessons learned about "Pragmatic Usability Planning for Interactive Media Applications in EU Projects" CONTESSA (Cross Media Publishing), CAMPIELLO (Tourist Information Systems), and iMEDIA (Intelligent Television Advertising). He regards contextual design as promoted by Beyer & Holtzblatt a conceptually good approach, but very labour intensive. 1-2 day site visits are not sufficient for user needs elicitation, as has been claimed. Project participants, including manager and UI designer, had attended a VNET5 workshop and received coaching support. VNET5 was considered an ideal partner, but the time for coaching per project seemes too limited. They learned from VNET5 that development team members themselves are responsible for usability! Approaches to user needs analysis that worked well were 1) asking the user partners in a project to provide available user data, 2) benchmarking of existing systems to make decisions about useful features, and 3) building rapid prototypes for new features and testing these with users. Feature lists and UML specifications are not considered sufficient for the specification of system requirements. The involvement of a developer in user requirements collection was found to improve the results of user needs analysis. Other pitfalls detected were discrepancies of user versus customer goals, and user versus consumer needs (pdf, 1154 KB).

Tapani Sarjakoski from the Finnish Geodetic Institute focused on a desk study of user requirements for mobile services utilizing topographic maps and on user testing in the GiMoDig project (Geospatial info-mobility service by real-time data-integration and generalisation). Studying literature helped the project to collect relevant information about users and their goals, about the context of use, dataset and technical requirements. User tests with existing topographic maps, carried out in the lab and in the field, provided additional user needs and experiences for planning and carrying out future user tests. For a more infrastructure oriented project like GiMoDig the desk study provided sufficient information for the definition of a number of application scenarios and use cases with varying criteria. The existing methodology is appropriate for the analysis of the needs of end-users. However, other methods are needed for the analysis of the needs of service providers (eg. performance and response time of the service)
(download (pdf, 13 MB) at: http://gimodig.fgi.fi/pub_deliverables/GiMoDig-INFORM-2003-March.pdf.

James Orwell from Kingston University, UK, reported about user requirements analysis in the INMOVE project (Intelligent Mobile Video Environment). User scenarios and personas were defined for end-user oriented applications (eg home care, car surveillance, football application) and user acceptance of these concepts was investigated with prospective users in focus group sessions. User feedback helped to modify the scenarios and to reject scenarios which the users did not like. Collecting and updating the user needs and requirements results in four separate tables caused a significant amount of paper work. The statement of the engineers point of view that "functional requirements means compulsory, non-functional means optional" was critised by the audience. It is essential for the success of consumer products to seriously take into account non-functional user requirements (pdf, 2674 KB).

Thanos Demiris and Michalis Karagiozidis from INTRACOM described user requirements analysis with online questionnaires carried out in the LoVEUS project (Location Aware Visually Enhanced Ubiquitous Services). Online Questionnaires were developed, 325 responses were collected, 286 of the responses were analysed. A difficulty encountered was that mainly "technology adopters" completed the online questionnaire. These were users with a specific profile: more male than female, young persons with a high level of education, skilled in using PC, PDA or mobile phone. In addition 60 customers were interviewed by telephone call. The difficulty with telephone interviews was that an extremely high percentage of customers refused to participate in the interview after the first questions that corresponded to application scenarios. Similar difficulties were encountered in projects PiSTE (Personalised, Emmersive Sports TV Experience) and MELISA (Multiplatform ePublishing for Leisure and Interactive Sports Advertising) (pdf, 6279 KB) and (pdf, 206 KB) .

Ada Pateli from ELTRUN, Athens, described the application of contexual inquiry methodology for capturing end-users requirements in the mEXPRESS project (mobile in-EXhibition PRovision of Electronic Support Services). Different types of users (visitors, exhibitors and organisors of exhibitions) were observed in their working environment (exhibition hall), short and long, in-depth interviews were carried out with different users. Contextual Inquiry is suitable for capturing user needs for innovative services when complex data in information rich environments with frequent user interactions have to be collected without loosing detail. A combination of different techniques (observation, questionnaires and interviews) with solid preparation and pilot test is considered useful for a Contextual Inquiry (pdf, 456 KB) .

Pavlos Vlachos from ELTRUN, presented how end users' attitudes towards mobile music services in Europe were investigated in the MUSICAL project (Multimedia Streaming of Interactive Content across mobiLe networks). Key success factors for MUSICAL could be detected with online questionnaires and in-depth interviews. The techniques were useful for example to learn about the willingness of music consumers and music professional to pay for MUSICAL, about important selection criteria, relevant functions requested by users, and national differences of music consumers (pdf, 168 KB) .

Conclusions

A broad range of techniques for user needs analysis is currently applied in different IST projects: desk studies and literature surveys, scenarios and personas, focus group analysis, use cases, also comprehensive contextual inquiry including a combination of several techniques like observation, interviews and questionnaires.

Efficient and effective techniques are preferred. On-line questionnaires seem to be an appropriate solution in this respect even though the responses come to a large extent from so-called "early adopters" or "technology adopters". This seems to be a bias which can be neglected.

Some projects clearly tried to build on results from preceeding projects by using, improving and extending previously used tailor-made questionnaires, by investigating the literature and by searching for available data from user partners and from other departments in the companies and organisations involved in a project.

Techniques which can be applied very early in the development process are given priority.

This was the last workshop conducted by the VNET5 team. It received positive feedback from the participants and demonstrated how user-centered thinking has gained strength in many organisations. Also, the expectations for continuing cooperation and expert support for competence building were noted during the event, and complemented by the responses received after the event.