Usability studies

I’ve been performing usability studies for different projects to evaluate products and services in diverse technology readiness levels. Below I give some examples of recent work performed where I applied UX, user-centred design and ergonomics research to understand how people interact with the systems in question.

Internal vehicle HMI

As part of the UK Autodrive project, partners Jaguar Land Rover and RDM/Aurrigo wanted to understand the interaction between occupants of automated vehicles and four types of interfaces. So our team designed a study to evaluate different configurations of screens comparing how they communicated the possible hazards in the environment (e.g. vulnerable road users), and vehicle behaviours (e.g. intended trajectory). These interfaces were presented in a fully automated vehicle tested by 25 participants in an indoor arena.

Surveys and interviews measured trust, usability and experience after users were driven by an automated low-speed pod. Participants experienced four types of interfaces, from a simple journey tracker to a windscreen-wide augmented reality (AR) interface which overlays hazards highlighted in the environment and the trajectory of the vehicle.

A combination of the survey and interview data showed a clear preference for the AR windscreen and an animated representation of the environment. The trust in the vehicle featuring these interfaces was significantly higher than pretrial measurements. However, some users questioned if they want to see this information all the time. One additional result was that some users felt motion sick when presented with the more engaging content. This paper provides recommendations for the design of interfaces with the potential to improve trust and user experience within highly automated vehicles. This study was previously published here:

Control interfaces

Another example of study include the testing of interfaces for controlling the vehicle, comparing personal smartphones, a tablet-type of interface fixed within the vehicle, and a larger display embedded above the windscreen of the vehicle. This study was published as part of the proceedings of the Automotive Interfaces conference, Auto-UI 2018. It can be seen via the link

External vehicle HMI

In another study we examined pedestrian attitudes and road-crossing intentions using these autonomous vehicle in an indoor arena. Two conceptual external human-machine interfaces were presented to display the vehicle’s manoeuvring intentions. One used regular indicators flashing amber LEDs around the wheel wings. The other presented projections on the road to tell exactly where the vehicle was going to turn and to indicate deceleration.

The arena where these tests were performed.

We collected objective measures (via surveys) and critical subjective opinions (via open-ended questions) regarding the interfaces in relation to two specific manoeuvring contexts. These provided insights into pedestrians’ safety concerns towards a vehicle where bi-directional communication with a driver is no longer possible, with suggestions for future vehicle HMI concepts. More information about this study can be found on the respective publication, part of the IEEE conference proceedings:

Train seat displays

A quick study was designed to provide an evaluation of two types of train seat displays: one the regular OLED, frequently found on board trains, and the new E-ink, which is used for other applications but not yet for displaying seating information. One intern helped conducting this study and reporting results.

Numerous factors influence the decisions on which display to use, such as costs or energy use. OLED displays tend to be more limited on size, font and characters displayed. E-ink allow new possibilities such as a variety of fonts and even images on these displays.

However, it’s important to consider human-factors aspects such as readability and user preferences. To provide some real-world insights into these issues we designed a pilot study to compare both displays. Participants were asked to read the information on both displays (in a counter-balanced order) give their impressions and respond to questions during a semi-structured interview process.

Usability study setting to compare displays

Results show that participants favour the traditional OLED display overall as it is easily noticeable in different light conditions. However, some aspects of the E-ink are preferred: it is easier to read and understand. Therefore, if the new E-ink is to be adopted in the context of train carriages, a few modifications are required to make them noticeable and accessible. This publication was a short paper presented at the Intelligent Transportation System conference, and the full paper can be accessed here:

Hardware and software integration

The final study of the CLoSeR project involved testing the whole system using a prototype of the train coach fitted with the proposed technology and a smartphone app. Users were invited to test the system in a lab at the University of Warwick.

I designed the tasks and measurements to understand user perceptions. Quantitative and qualitative methods were applied, and the results informed the (re)design of the technology. A number of recommendations was shared with the development team, who performed the changes prior to commercial demonstration. I added below images of the hardware evaluated during this study. Due to IP and comercial sensitivity, further details of the system cannot be shared.

Other observational techniques I performed recently include shadowing, with train crew, and unobtrusive user observation, when participants are unaware of the data collection.