Agile Prototyping for user EXperience







  • New paper accepted for publication at Int. J. Human-Computer Studies: Prototyping and Analysing Ubiquitous Computing Environments using Multiple Layers.
  • New paper at EAI Endorsed Transactions on Ambient Systems, A Virtual Environment based Serious Game to Support Health Education.
  • Two new MSc thesis: 3D Virtual Environments' Generation by Tiago Gomes and Responsive Web Design by Ana Isabel Sampaio.
  • Award: José Luís Silva's thesis won the Best Iberian Thesis Award! Congratulations.
  • New MSc thesis: Simulating Ubiquitous Computing Environments by Samuel Moreira
  • New PhD: Rapid Prototyping of Ubiquitous Computing Environments by José Luís Silva. Congratulations José!
  • 2nd year Project workshop happening November 6-7, 2012.
  • Two new papers at SEW: Guidelines for Modelling Reactive Systems with Coloured Petri Nets by Gonçalves and Fernandes; and A Java based PSM/PIM and pattern inference approach by Couto el al.
  • New paper at MOMPES: MapIt: A model based pattern recovery tool by Couto et al.

Project Description

Plan and Methods

The core problem addressed by this proposal is that of being able to test design ideas for ubiquitous environments during the early phases of the design process. This implies that the analysis must be performed without the need to fully develop and install the envisaged system on location. Our proposal is to use a 3D Application Server as the basis for a prototyping framework that will enable agile development of simulations of the ubiquitous environment. This framework will enable different degrees of fidelity for the prototypes, from simple desktop simulations, to fully immersive experiences in a CAVE environment.

Having such a prototyping framework available will affect the development of ubiquitous systems by: a) detecting problems in the design of the systems much earlier in design and development; b) making it possible to test the system without access to the actual installation site. Both contribute to reductions in the cost of development and testing, while improving the quality of the final product.

The core of the work will be based at Minho/CCTC and will be supported by a PhD student (with grant SFRH/BD/41179/2007 from FCT) working under the co-supervision of the PI at Minho and Michael Harrison based at Newcastle but an honorary professor at Minho. Minho/CCTC has a strong reputation for modelling and analysis. A CAVE at the Computer Graphics Center in Minho will be made available to the project (at a cost). Minho/Algoritmi brings to the project expertise in ubiquitous systems development. The consultant, Jorge Santos from CPsi (Minho) provides expertise in Psychology, and in particular in user testing within the CAVE at CCG.

The work plan is divided into four phases. The phases are organized so that the work starts with more conceptual work (reviewing and analysis), and gradually moves into more practical work (leading to an applied case study).

Phase 1: Analysis

A modelling approach based on CPN [27] will be developed. The focus will be on modelling for rapid prototyping, and the modelling of user context. It is envisaged that languages will be needed for modelling both the architectural and behavioural aspects of the ubiquitous systems. It is envisaged that further development of the approach will lead to the use of a combination of notations. The selection of the languages will be preceded by an analysis of the requirements for the prototyping of ubiquitous systems (system centric view), and the requirements for experience prototyping (user centric view).

Associated tasks: Analysis of requirements for experience prototypes; Development of modelling approach.

Phase 2: Development of the 3D prototyping framework

A framework will be developed, enabling the simulation of environments enhanced with sensors and displays in which the user can move and interact, both explicitly and implicitly, to access services. Interaction will be supported both through the virtual environment and through personal devices connected to the system via Bluetooth.

This phase encompasses two main challenges: (a) developing a framework to support the animation of the prototypes; (b) developing a prototype generation engine that is capable of building ubiquitous environments from models expressed in the selected modelling notations as described in phase 1.

The framework will be developed in OpenSimulator. A 3D application server such as OpenSimulator will make it easier to create the virtual environments, and has the added advantage of making it easy to deploy the prototypes to distributed users. When combined with a desktop computer to navigate the world, it is also a low cost approach since hardware and software requirements are low. The desktop approach however, makes it harder to create a sense of presence. Presence will be addressed by connecting the system developed on a desktop to external devices and to a CAVE system. This will enable more realistic prototypes of the prototyped environments with the aim of making it easier to create a sense of “being there”.

Although the development of the OpenSimulator based prototyping framework is crucial, development of the other aspects can start independently from it if needed, since the major technical challenge is connecting OpenSimulator to external systems. A CAVE is available at Minho.

Two issues must be addressed in creating a prototype: (i) generating the skeleton environment with mock-ups of the devices exhibiting adequate behaviour; (ii) configuring the environment to a more real life appearance.

The initial version of the environment should be enough to perform initial evaluations of a design. Configuring the environment amounts to defining in detail the physical objects present in the prototype. This will allow for a more realistic user experience. Early versions of the system will be based on a created catalogue of physical objects. The early worlds will be based on “off-the-shelf” components. The objective is to use intelligent algorithms to generate candidate objects on demand.

The case study used in phase 1 and 2 will be used to test the framework.

Associated tasks: Development of the basic animation framework; Development of the prototype generator; Development of the configuration generator (MSc); Extension to the CAVE (MSc); Extension to external devices (MSc);

Phase 3: Assessment

In this phase a study will be carried out regarding techniques for evaluating user experience within the prototypes. It is envisaged that both free and guided exploration of the constructed environments will be used. Approaches must be selected to determine user reaction when exposed to the environment to establish the fidelity of the prototype environment.

An existing ubiquitous computing system will be used as a case study. This will be selected from system implemented at Algoritmi. The case study will be carried out to assess the use of the developed infrastructure. The goal of the exercise will be three fold: (1) to assess the expressiveness of the selected modelling notation as a tool for ubiquitous computing systems modelling; (2) to assess the capabilities of the infrastructures as a tool for experience centred agile prototyping for ambient intelligence; (3) to explore the fidelity of these prototypes as a factor in evaluating the eventual human (usability and social) impact of the design. This will entail an analysis of the feasibility and effort involved in creating the prototypes, as well as a comparative analysis of user experience in the real system and in the prototypes. In parallel with this phase, but outside the scope of the project, Harrison will explore the expressiveness of the notation in terms of its capability for verification.

Associated tasks: Assessment


T1 - RAMP: Requirements for ambient and mobile system prototypes

The goal of this task is to build an understanding of the requirements for a prototype that will enable users to experience the ubiquitous system sufficiently to be able to provide information about its suitability in a proposed target environment.

The analysis will be performed at two complementary levels. On the one hand, we need to understand what features the prototype should have to enable a sense of experiencing the eventual target environment. On the other hand, we need an understanding of the type of information, and software infrastructure needed to build prototypes that exhibit the required features.

Following from the state of the art produced previously by members of the team, it is a basic assumption of the project that at least some sort of virtual world representation of the envisaged system will be needed. This analysis will proceed through a review of relevant literature and through the definition of a case study. At this stage a library space, enriched with public displays and touch screens, RFID tags to determine the position of books and users, and capable of interacting with the users’ personal devices is envisaged. The infrastructure should enable users to more easily find the book of their choice, as well as provide them with tailored information regarding the library’s activities and contents.

T2 - DOMA: Development of modelling approach

The goal of this task is to develop the modelling approach for rapid prototype generation.

The approach should be expressive enough to make the generation of prototypes as automatable as possible, but abstract enough to enable a cycle of the rapid development, and throw away of different models cost-effective. The work carried out in the context of this task will be informed by the results of task T1.

The adoption of existing modelling languages will be preferred to the development of new languages. A preliminary literature review (mentioned in [22]) has enabled us to identify a number of candidate languages for the modelling and prototyping of ubiquitous systems. Based on the preliminary study it is envisaged that a combination of languages will be needed to cover both the architectural and behavioural levels of the systems. A choice has already bean made to use CPN as the basis for the behavioural models.

One aspect to consider is how to best integrate the notion of context into the modelling approach being devised.

T3 - PFD: Prototyping framework development

This task will develop a framework to support the simulation of ubiquitous systems. Prototypes will consist of virtual environments in which the users can navigate, and where they will interact with available (virtual) devices.

Framework development will be informed by the requirements identified in task T1.

A 3D application server will be used to fast track the development of the framework. This approach had the added value of enabling easy web dissemination of the prototypes, and of being relatively inexpensive in terms of required hardware. A typical desktop or laptop computer is capable of running the required software satisfactorily.

At this stage OpenSimulator has been identified as the best candidate. The reason for this is that it is an open source system, enabling a much wider range of possibilities regarding its configuration and extension. The main issues to consider are related to fact that the system is still under development, and the lack of adequate documentation. These risks are minimised by the fact that expertise exists in the team regarding development for OpenSimulator.

An alternative could be the adoption of SecondLife. This system is commercial, which would increase the cost, and inevitably the practicality, of generalising the applicability of the approach. More relevant, however, is the fact that, because it is proprietary, SecondLife is a relatively closed system. This would make developing the framework harder to achieve.

The developed framework will support both implicit and explicit interaction. Implicit interaction will happen through virtual sensors that capture, for example, the position of the user in the space. Explicit interaction implies the availability of virtual devices presented in the simulation, for example, touch screens or simulated portable devices.

T4 - PGD: Prototypes generator development

This task will be responsible for producing the tool that will enable the animation of the models (expressed in the modelling notation selected in task T2) in the prototyping framework (developed in task T3). Development of the tool will be informed by the requirements identified in task T1.

The tool will be responsible for generating a skeleton environment with mock-ups of the devices exhibiting adequate behaviour. At this stage a library of available objects will be used to ease the generation of the environment. Behaviour of the object can be partially pre-defined, but ultimately it will be specified in the model and will have to be generated. Detailed physical layout is not expected to be present in the models.

The generator will use a combination of modules programmed into the server’s back-end, with the already existing capabilities of interactively arranging objects in the world (provided by the 3D application server) to generate the prototypes in a semi-automatic fashion.

These prototypes will be enough to perform initial evaluations of prototypes. The case study defined in task T1 will be used to test the approach.

T5 - CGD: Configurations generator development

This task will be responsible for developing approaches and tools that will help generate prototypes with a more real life appearance. The results from tasks T2 and T3 will be used as a starting point.

To provide a more realistic appearance to the prototypes without requiring significant effort from designers, this task will explore interactive evolutionary computation approaches to 3D design and behaviour programming. The focus of this task will be on finding suitable parametric models of the objects that can be modelled in relation to the previous tasks. A focus will be to establish an understanding of the performance of different algorithms in the search for objects that are adequate for the user and the automatically-generated smart space configuration.

The case study from previous phases will continue to be used. In particular, this task will select a number of different objects from this case study for which a parametric model will be identified. For each object we will evaluate the outcome of the evolutionary computation approaches regarding the number of iterations to find a suitable object and the realism of the object found.

T6 - CD: CAVE deployment

The desktop approach, developed in tasks T3 and T4, makes it possible to generate prototypes rapidly, and to test them more economically. However, creating a sense of presence in such a setting might prove difficult. This task will be responsible for linking the prototyping framework to a CAVE system. This will enable a more immersive experience of the prototype, making it easier to create a sense of “being there”.

Although this task follows from tasks T3 and T4, development can in fact start independently from it if needed, since the major technical challenge is connecting OpenSimulator to the CAVE systems. The use of a programmatic interface to OpenSimulator enables the creation of a number of avatars that will provide the views to project on the CAVE.

A CAVE is available (at a cost) at Minho. The CAVE uses VR Juggler ( which is a freely available open source virtual reality application development framework. Being available free of charge, its adoption by the project signifies a considerable cost reduction, when compared with the cost involved in acquiring licences for a commercial framework.

Given the cost of CAVE time, we need to maximize the utility of the time spent there. A triple head setup will also be put in place to simulate a CAVE scenario. This will allow us to carry out development and testing outside of the CAVE.

T7 - IMD: Integration with mobile devices

This task will be responsible for developing a solution to connect external devices to the virtual environments generated using the tool developed in tasks T3 and T4. As for task T7, the goal is to create conditions to enable a more realistic user experience. In this case we want to enable users of the prototypes (either at a desktop or at a CAVE) to interact with them via their own portable devices (see, for example, [24]). This will enable more realistic prototypes of the prototyped environments, contributing (especially when in a CAVE setting) to the creation of a sense of “being there”. Even in the case of a desktop setting, being able to use own portable devices will increase the ease of use and facilitate the interaction with the simulation.

As for task T6, although this task follows from tasks T3 and T4, development can start independently from it if needed. In this case, the major technical challenge is connecting OpenSimulator to the mobile systems. The connection will be achieved via Bluetooth. Members of the research team have significant practical experience at developing this type of system.

T8 - A: Assessment

There are two aspects to the Assessment task. The first is concerned with the expressiveness of tool (Tn.1) and will deal with the ability of the tool to provide models of realistic technologically enhanced environments. The second will be concerned with evaluating the proposed design with users.

Sub-Task T8.1
This evaluation will be achieved through the development of example systems. The first example that will be used to motivate the design of the tool will be a smart library. The evaluation will be based on an analysis of features: (1) the ease with which different notions of context could be expressed and the clarity and readability of the relevant elements of the model; (2) the availability of appropriate objects required to construct the environment; (3) the facility of the system to support the building of the environment based on binding the model to the chosen objects; (4) the facility of the system to support differing user perspectives. A second example will be chosen to achieve comparison with the original model. A candidate for the second example is a hospital out-patients guidance system, combined with accident and emergency. We will however choose from a number of alternatives as the project evolves.

Sub-Task T8.2
This evaluation will be achieved in stages.

Stage 1 (desktop with the library system)
A think-aloud approach will be used first to explore scenarios in relation to three conditions within the desk-top environment: (a) desk-top system using a disembodied view; (b) desk-top system with one user embodied within the system; (c) multiple users with one user embodied within the system.
Stage 2 (CAVE with the library system)
This will repeat (b) and (c) above. The aim of stages 1 and 2 is to get constructive formative feedback both about the design of the artifacts of the environment and the environment itself. Two subjects will be used in each condition and think-aloud feedback will be recorded and analysed, identifying significant issues in both the design of the simulations and the design of the system itself. Particular concerns will be to identify how well the users are able to “visualize” the experience that they would have of the real system.
Stage 3 (desktop and CAVE with second example)
Here users with recent experience of the currently existing system will be chosen (for example hospital out-patients). They will be asked to describe their recent experience in detail. The detail will form the basis for a scenario that will be used to explore the desktop environment. Hence the procedure will be to invite the subject to relive their experience using the new environment. The aim is to find ten subjects and to split them into two groups of five: one using the immersive version of the desktop system and the other using the CAVE system. A post-simulation questionnaire and interview will be used to explore the experience that they had with the new system in the two conditions. In addition we will ask the subjects to use “simulated” cultural probes [29] (they will use a clicker to indicate significant events) and will be asked afterwards in a run-through of their simulation what the significance of each event was.

It should be noted that controlled experiments are seen as inappropriate in the context of this project. Our aim is to derive formative qualitative data that can inform our understanding of the value of the design to explore the development of these environments.

-- JoseCampos - 09 Feb 2011

r2 - 11 Feb 2011 - 00:47:39 - JoseCampos
This site is powered by the TWiki collaboration platform Copyright © by the contributing authors. Ideas, requests, problems? Send feedback.
Syndicate this site RSSATOM