The voice of art

a artificial intelligence project in partnership with three major museums in Latin America, Pinacoteca and IBM

By Flavia Goulart

November 25, 2020

The voice of art

IBM & Pinacoteca Museum

Project Description

With a market estimated in US$ 1.2 billion for 2017, artificial intelligence (A.I.) is the new technology frontier. IBM owns the most advanced A.I. platform in the world – Watson –, which has been disrupting industries such as healthcare, education and finance. However, in Brazil, people’s familiarity with A.I. is still superficial when compared to that of other countries. Also, brands such as Microsoft and Google have a high awareness in the category, threatening IBM despite its pioneering. So, positioning the IBM brand as the leader in A.I. and the only one that can bring it to life have become a priority, specially this year, when IBM celebrates its centennial in Brazil. To do so, a traditional advertising campaign wouldn’t be enough. A memorable experience with Watson would be crucial to make people feel the impact A.I. and Watson can bring to their lives and how it can transform businesses.

And so, The Voice of Art was born—an experience that developed an app (a cognitive assistant) capable of “talking” with the museum audience about seven artworks from the collection.

Details

Role: Senior/ staff User experience
Company: Ogilvy & Mather agency
Client: IBM
Project: The Voice of Art

Overview

72% of Brazilians have never been to museums nor art institutes because they do not relate to or understand art. For the launch of Watson in Brazil, IBM collaborated with Ogilvy to create “The Voice of Art” project, an interactive guide that allowed people to have conversations with art pieces housed in Pinacoteca, São Paulo’s oldest museum, known for its innovative vein. Initially for two months, Pinacoteca’s visitors could use an exclusive app to literally talk to seven art pieces of the collection, asking them whatever they wanted, the way they wanted; the response was given by Watson through its Brazilian voice in real time and natural language, explaining subjects such as the history behind the piece, art techniques and styling, and even the relationship with current topics. It was a creative and unprecedented idea that not only brought museum visitors’ experience to a new level and generated awareness to Watson: it also made Brazilians become more open to and interested in art.

The Challenge

Design a solution for a broad audience aged from 6 to 70 years old.
Introduce the app in the context of museum visitation without interfering with the regular flow of visitors.
Training the watson as many question as possible, even distinguishing the age of the person who’s asking.
Design a mobile app to help visitors and guide them to have the best experience.

My role

I was the main senior UX design responsible for the role project. We did research, studied the audience, prototyped based on the results of our findings, tested a lot and constantly iterated during the process. We spent 5 months from the beginning of studying to the App launch.

Main tasks

1. Understand how watson worked (developers).
2. Research about the pinacoteca values and objetives, who the visitors were (how they behaved, what works they saw first, if they talked to each other), and visit the museum to find out where and how we should guide visitors
3. Plan and define product scope.
4. Plan and apply the user research process to understand the needs of the museum’s audience.
5. Create an intuitive product flows, to define interaction models, user’s task flows and design the interface.
6. Execution prioritizing user experience and Pinacoteca and IBM brand.

Project Schedule

We‘ve developed an action plan that splitted the work into:

User Research
Insights and ideas for the app
Prototyping
Tests
Iteration for final solution

User Research

The biggest challenge of this project was designing a product that could serve a broad audience from 6 to 70 years old, since we’re talking about all visitors of the Pinacoteca museum. To begin with, we did a qualitative research by interviewing 25 people, a considerable amount based on the extensive age range.

Learnings & Data

The goal was to understand users’ behavior regarding the visiting experience at the museum, by mapping their journey and needs. Furthermore, we’ve studied their behavior while interacting with daily apps so that we could design an experience as familiar as possible.

Audience Insights

40-70 years: Feel a barrier to museums. Prefer audio guides offered at the start.
6-12 years: Exploration experience. Get bored with pictures, prefer interactive pieces.
General: Prefer indications on where to start.

Museum Data (2016)

Educative Audience: +42K people
Total Audience: +325K people
Peak: Weekends (4x more visitors)

Design Sprint & Solution

The strategy was putting all Watson’s cognitive power into a mobile app available to visitors of the Pinacoteca Museum. This application uses several Watson APIs to recognize questions, decode intent, and deliver specific answers.

In addition, we’ve incorporated a system at the museum building that recognizes the visitors’ presence in front of the art pieces to begin the interaction in the app without visually interfering in the art pieces.

App Experience

Considering all learnings, we developed the wireframe with 4 main intentions:

Create a great onboarding process
Locate and recognize the art piece in the museum
Facilitate interaction with the art piece
Evaluate Watson’s answers to improve its cognition

Art Piece Location & Recognition

Questions Recognition

The main goal was to encourage the user to talk to the art piece, so we’ve highlighted the question button right on the interface.

Mapping messages (Beacons)

We’ve classified the beacons into 3 functions: Location indicator, Art pieces, and Gateway/Exit.

Testing & Iteration

We’ve tested and iterated the app each week for 2 months. The main lessons were that people struggle with maps (preferring photos and room names) and don't like to follow strict scripts.