This article is part of a Special Issue on The Social Value of European Research on Media Accessibility.

Research on interactive content consumption has been intense in the last years. As proof of evidence, previous works have targeted at integrating accessibility features, companion screens, and immersive content in interactive TV services. All these features provide an added value, but have not been investigated in a jointly manner yet. To that end, Immersive Accessibility (ImAc) is a European H2020 project that is exploring how accessibility services can be efficiently integrated with immersive media. In particular, ImAc aims to ensure that immersive media experiences, involving the consumption of 360º video and spatial audio, probably in combination with traditional media formats, are inclusive across different languages, addressing the needs not only of those with hearing and low vision problems, but also of people with cognitive or learning difficulties, newcomers, people with low literacy, and the aged, in an interactive and personalized manner.

Many challenges, like the following ones, need to be overcome to achieve the targeted goals:

• A key intrinsic characteristic of Virtual Reality (VR) content, like 360º video, is the freedom to explore around the omnidirectional environment. Therefore, viewers could miss the information from scenes where the main action takes place, if these scenes are outside their current field of view. This means that the presentation of accessibility content (subtitles, audio description, and sign language interpreting) in 360º environments does not have to only consider the temporal dimension (e.g. the presentation periods), but also the spatial dimension (e.g. in what direction the actions take place).
• Immersive content is commonly presented in a 3D omnidirectional projection format. Likewise, the field of view when using VR devices, like Head Mounted Displays (HMDs), is typically limited. Therefore, appropriate User Interfaces (UIs) need to be designed to maximize usability and a comfortable viewing experience, while being accessible.
• Although interaction modalities when using classical consumption devices (TVs, laptops…) are well known, this is not the case yet for immersive consumption devices. Appropriate interaction modalities, like head movement sensors and joysticks, and of assistive technologies, like voice control and zoom features, need to be adopted.

All these open issues are being addressed in ImAc in order to provide the most appropriate access features, presentation modes, personalization settings and assistive methods, according to the particular needs and/or preferences of the audience. To efficiently achieve this, the project has adopted a user-centric methodology, in which end-users, professionals and stakeholders are involved at every stage of the project through the organization of workshops, focus groups, tests, and the attendance to events. This allows accurately gathering the requirements to be provided. The insights from the user-centric activities also determine the necessary technological solutions to meet these requirements, and the specific services and scenarios to be provided.

End-to-End ImAc Platform

ImAc is developing an end-to-end platform to allow the integration of immersive and accessibility content in current broadcast-related services. The platform comprises four parts, including the necessary components and steps from media production to media consumption.

Content Production: (web-based) tools for the production and editing of accessibility content, including subtitles, audio description and sign language.

Content Provider: extension of the infrastructures and workflows currently used by content providers to accommodate the contributions from ImAc. This includes the development of an Accessibility Content Manager (ACM) to upload the immersive contents, manage the creation of accessibility contents and trigger the preparation of content for being distributed.

Content Preparation & Distribution: preparation of content (e.g. format conversion, encoding, signalling…) for an efficient distribution via broadcast and broadband networks.

Content Consumption: a web-based player for the presentation of immersive and accessibility content in an interactive and personalized manner. The player can be run on traditional consumer devices (e.g. Connected TVs, laptops, tablets and smartphones) and on VR devices (e.g. HMDs), and can be integrated in multi-screen scenarios. A demo video can be watched here.

All technological contributions have been designed with the premise of guaranteeing backward-compliance with current technologies, formats, infrastructures and practices in the broadcast industry. This will maximize re-usability, interoperability and the changes of successful deployment and exploitation. ImAc is indeed actively contributing to standardization of technologies, formats and recommendations in different international bodies, like the World Wide Web Consortium (W3C), Moving Picture Experts Group (MPEG) and International Organization for Standardization (ISO). Finally, the conducted pilot and dissemination actions allow validating its contributions, but also refining them based on the obtained results and gathered feedback. Finally, the conducted pilot and dissemination actions, together with the launched exploitation strategies, allow validating the project’s contributions, but also refining them based on the obtained results and gathered feedback.

Pilots

The evaluation of the developed technological contributions, and of the created contents and services, becomes essential to validate the provided benefits. In this context, ImAc has planned diverse pilot actions. On the one hand, the content management and editing tools are being evaluated by professionals, mainly in terms of performance, usability and usefulness of their features. On the other hand, the created contents and the features provided by the player are being evaluated via end-user tests, considering standard immersion questionnaires and questions on preferences. In a first iteration, different aspects are being evaluated, like the comfortable field of view, the usability of the player, the most appropriate presentation modes for each access service; the use of assistive technologies, the desired personalization features, etc. With the lessons learned, open pilots will be launched in a second iteration, via both the websites of broadcasters and their TV programmes, which will be augmented with accessible immersive content via companion screens.

By Mario Montagud (i2CAT Foundation & University of Valencia, Spain)

Go back to the Special Issue: The Social Value of European Research on Media Accessibility!