Innovate or stagnate: Disrupting the conventional audio guide
Shelley Mannion, The British Museum, UK, Amalia Sabiescu, RMIT Europe, Spain, William Robinson, British Museum, UK
AbstractInnovation is crucial to the success of digital in museums, but innovating within a well-established product genre like the audio guide is potentially risky. New approaches can challenge existing business models, jeopardise revenue streams, and undermine visitor satisfaction. In 2015, the British Museum tested some hypotheses about how to improve exhibition audio guides through two large-scale experiments. This paper shares the results of those experiments and the experiences of over one hundred visitors and staff who participated in them. The findings are discussed within the wider context of the Museum’s new digital strategy that emphasises speed to market and continuous optimisation. We draw on Christensen’s model of innovation to make recommendations about how to evolve a product strategy for audio guides that appeals to existing audiences and attracts new ones.
Keywords: audio guide, mobile, guide, qualitative, evaluation, product, digital, strategy, disruptive, innovation
In January 2015, the British Museum launched a new digital strategy with three main goals: generating revenue, increasing reach, and building relationships. The focus on revenue prompted a hard look at our business model for audio guides, a key visitor-facing product. Our permanent-collection audio guide is a growing business, but the value proposition for exhibition guides is less clear. In recent years, we have seen little change in the patterns of usage and sales of the guides. Take-up rates are roughly the same for every exhibition; evaluations from visitors are consistently positive, but not enthusiastic. With production costs rising and exhibition budgets stretched to the limit, our challenge was to find ways of invigorating audio guides to generate more revenue while maintaining quality. To tackle this challenge, we conducted two experiments in large exhibitions and evaluated the results.
We discovered that innovating within an existing product genre like audio guides is risky. It is especially hard to offer new formats that satisfy frequent users and also appeal to new and occasional users. Frequent users have strong expectations about what a guide should be and are critical of products that disrupt the familiar model. Christensen’s innovation theory (1997) offers a way to understand these findings. He defines two types of innovation: sustaining and disruptive. Sustaining innovations improve on existing products without jeopardising the existing customer base. Disruptive innovations attack an established market from the outside by providing a similar product for a cheaper price. Our experiments provided examples of both types, as well as insight into how to evolve our product strategy for the guides.
2. What we did
We chose two upcoming exhibitions to test new approaches to our audio guides. Defining beauty: The body in ancient Greek art ran from March to July 2015. It was the year’s blockbuster exhibition. Indigenous Australia: Enduring civilisation ran from April to August 2015.
We produced two versions of the audio guide for each exhibition:
- In Defining beauty, we tested the format of individual stops by recording contributors in conversation with the curator. This approach was inspired by earlier work at the Portland Art Museum (Harris et al., 2010). We invited experts from different fields, including several artists, to give their perspectives on specific objects. Lengthy recorded conversations were edited down to two- to three-minute stops in the first version of the guide. In the second version, we added short, narrated introductions to each stop. The conversations themselves remained the same in both versions.
- In Indigenous Australia, we tested the overall format of the guide. Version 1 was a conventional object-based format with fifteen two- to three-minute stops. Version 2 was organised around themes instead of objects. It had seven stops, each five to seven minutes long. They introduced an entire section, then homed in on a few objects within that section. Both versions had the same narrator and included the same or similar contributions from artists and experts chosen by the curator.
In all, we conducted five qualitative evaluations. We evaluated the first version of the Defining beauty guide with ten visitors and twenty staff members, made changes based on the results, and then evaluated the second version with thirty-five people. In Indigenous Australia, the object-based and thematic guides were evaluated with twenty-five and twenty-two visitors, respectively.
To ensure accurate comparisons, the same scriptwriters, production team, and original recordings were used for each version. All versions were delivered on the same software and hardware. Nearly everyone, including self-identified technophobes, rated the software “very easy to use.” Not one reported technical problems, which meant we could concentrate exclusively on visitors’ reactions to the format and content of the guides, rather than the usability of the device and its interface.
3. What we found
Highs and lows
Of all the guides we produced, the thematic guide for Indigenous Australia was the most popular with visitors. It was appreciated by three categories of guide takers: frequent, occasional, and first-timers. Version 1 of the Defining beauty guide was the least successful. Frequent guide takers were especially critical. First-timers and occasional guide takers were more appreciative, but overall, the conversational format challenged visitors to the point that they found it difficult to see its benefits.
We learned much about visitors’ expectations of what an audio guide should be. The strength and specificity of these expectations surprised us. Frequent guide takers carry with them a lifetime of experience using audio guides. Deviating from the norm forces them out of their comfort zone. It is difficult to experiment without risking their satisfaction and, by extension, jeopardising your existing user base. On the flip side, occasional and first-time guide takers have fewer preconceptions and are open to new formats.
The question “What, for you, is the purpose of an audio guide in a museum exhibition?” yielded the most interesting responses. One answer was repeated like a refrain: “To give more information than what is on the labels.” Visitors went on to articulate the strategies they used to combine listening, reading, and looking. This revealed an unexpected (for us at least) level of sophistication in how they consume our interpretive offerings. Organisational workflows may make it hard to achieve seamless integration between audio guide, written word, and display, but savvy visitors are attentive to and often critical of how well these different interpretive elements complement each other.
Time is against us
Deploying new versions within the tight time frame of a three- to four-month exhibition is hard. In Defining beauty, the second version of the guide required scripting and recording of new material. We struggled to release it before the exhibition closed. Indigenous Australia presented different challenges: producing two complete, polished guides and replacing all the in-gallery signposting. This incurred extra costs and demanded support from the exhibition team during an extraordinarily busy period.
Many British Museum visitors use audio guides regularly in our exhibitions and at other venues. As a result, they had strong expectations about how the guides should work. These influenced their overall satisfaction and openness to new formats. Expectations operated on three levels: product, exhibition, and institution.
Visitors thought that audio guides should:
- Provide more information than what is on the labels and not repeat them
- Provide deep background and contextual information on objects, ideally through an interface with multiple layers of commentaries that visitors can choose to consume or ignore, depending on their interest
- Not overwhelm with too much information
- Have short commentaries, about two minutes long
- Have plenty of stops (twelve stops in Defining beauty was too few; fifteen stops in the object-based guide for Indigenous Australia was about right)
- Contain the voice of the curator and other recognised experts
- Add the human element by including diverse voices
- Be cheap: 23 percent of visitors to Indigenous Australia said they were aware of the audio guide but decided not to take it because of the price
Visitors form expectations about the exhibition based on marketing they have seen, articles in the press or social media, and conversations with friends. Reviews in the press were an important driver for Defining beauty. The top motivations for visiting were intellectual: to improve their own knowledge and understanding (65 percent) and to gain deeper insight into the subject (54 percent). As a result, visitors expected a higher standard of information than the permanent collection (British Museum, 2015a).
One of the reasons I came was because it was curated and [therefore] you knew it was carefully chosen, and all in one place—it actually makes it much more accessible than visiting a museum. —Focus group participant
The conversational format of the audio guide was a hard sell in this context. Visitors complained that they did not hear enough from the curator and that the content was not factual or intellectually rigorous enough. These problems were compounded by the fact that nearly 20 percent of visitors (roughly twice that of a typical British Museum exhibition) said they had specialist knowledge of the topic, whereas the guide was pitched for visitors with general knowledge.
Indigenous Australia visitors were also intellectually motivated: 73 percent wanted to improve their own knowledge, and 59 percent hoped to gain deeper insight into the subject (British Museum, 2015b). Like Defining beauty attendees, they had high expectations of the depth and breadth of information the exhibition would contain. Both versions of the audio guide took a conventional approach to content. They featured a narrator and contributing remarks from a range of experts and artists, many of them from aboriginal communities. In this respect, they satisfied visitors’ intellectual expectations. The thematic guide was also successful in addressing the desire of many visitors for a moving narrative: 18 percent expected to be moved emotionally by the exhibition.
Visitors expected the British Museum to provide more written interpretation than art museums. (Note: The shorthand references that identify visitor quotes in this paper have three parts: exhibition abbreviation, stage 1 or 2, interviewee number.)
When you go round an art gallery you don’t always get the information. These [interpretive] boards are very good… they were particularly informative and not too long or long winded. But most places you go to you don’t get anything else, so you get all the background with the audio guide. But actually it was mostly covered with the boards in this exhibition. (IA-S2-22)
In an earlier study (Mannion, et al., 2015), we found this made some visitors less likely to take an audio guide because they were confident the panels and labels contained enough information. If they did take a guide, they were likely to combine reading and listening. The implication is that the design of audio guide content must take written interpretation into account in order to provide a coordinated approach across channels.
5. Conversation in Defining beauty
The conversational format of the Defining beauty guide challenged frequent guide takers who wanted a format they have come to expect from previous guides. For them, dialogue signalled subjective opinion rather than objective fact. They dismissed the commentary delivered through conversation as merely ideas, opinion, and conjecture. Our interviewees had strikingly similar complaints to the thirteen respondents in the Portland Art Museum study (Harris et al., 2010), who disliked the conversational approach.
|British Museum: Defining beauty||Portland Art Museum|
It was confusing, why were these people invited to speak, and what is it to do with me?
I would much more prefer a fact-based guide.
It’s OK but not for me.
It was too shallow. I prefer discussion in depth with relevant people (e.g., the curator).
This guide had more ideas rather than factual background information. I would have liked more of that.
|It was too wishy-washy. I would prefer one person giving art historical information.
The eavesdropping didn’t work for me.
I prefer an expert—knowing for sure that something is something.
OK. I wondered if they were professionals. Wondered if it were factual or no.
I prefer an expert talking.
I don’t like banter. The speakers need to look up information before recording. I could speculate, too.
Table 1: visitor criticisms of conversation
Our sample size was smaller (forty-five versus one hundred in the Portland study), but we had a higher percentage of visitors—nine of ten in the first version of the guide—who expressed these concerns. The number of complaints dropped dramatically in version 2 after narrated introductions were added to provide a more conventional frame for each stop. This minor change shifted visitors’ perceptions and reassured them that the information was trustworthy.
Credibility of speakers
The authenticity of the contributors was another concern in version 1 of the Defining beauty guide. Even though the curator introduced them with their professional titles, nearly all the respondents questioned their credentials.
I really want commentaries from scholars, from people who have knowledge, not from fashionable figures that are not professionals. What I appreciate is scholarship. As a non-professional or amateur I want to hear from someone who has devoted his life to [the subject]. I want to get access to their years of sweat. (DB-S2-22)
I don’t want people who are not experts in the field to waste my time. I am not interested in their opinion. (DB-S2-32)
The irony is that all of the contributors were hand-picked by the curator because of their knowledge and expertise. Five of the ten are respected academics specialising in subject of the exhibition. Others included an acclaimed neurobiologist, an editor of a best-selling magazine and three accomplished artists. Although many guide takers doubted the qualifications of the academic contributors, artists received little scrutiny. Visitors did not question their professional pedigrees and most found their commentaries insightful: “I liked hearing an actual sculptor talking, this made me look differently at things and understand things that would be difficult to imagine.” (DB-S2-01) This suggests certain types of experts may be a more natural fit for conversation.
To address the credibility issue, we replaced the curator’s original introductions of the contributors with elaborate ones read by the narrator. These emphasized their professional titles and institutional affiliations and explained briefly why they had been selected to contribute. These introductions made visitors much more receptive and reviews of the second version of the guide were much improved.
Participants in the Portland study (Harris et al., 2010) also queried contributors’ expertise, even though “the majority of the speakers were… experts by anyone’s reckoning: curators and art historians.” Not all introduced themselves, however, and the authors speculated whether adding introductions might help. Our findings confirm that introductions are essential in establishing credibility of speakers. They should be emphatic and, if appropriate, spoken by a third party rather than the speakers themselves.
Strengths of conversation
Once visitors were open to the conversational format, they articulated three benefits:
- Allows for contradictory views
I enjoyed the conversational style, at the beginning especially, when they contradicted each other. It is good to have different views. (DB-S2-11)
- Encourages reflection, prompts visitors to be active rather than passive
I liked it. It makes you think. It makes you think whether to agree or not. (DB-S2-17)
- Less formal and more relaxed than a conventional guide
It was good. The conversations were funny, incomplete in interesting ways. (DB-S2-28)
The Portland study demonstrated conversation’s potential to draw visitors in: “[Some] remarked on the informality of the interactions and articulated the ways that informality and the back and forth, made them feel included…” None of the forty-five visitors we interviewed in Defining beauty expressed this. It is possible they were too preoccupied with their expectations to fully embrace what the format offered. Perhaps our conversations lacked the polish that Harris and Zucker perfected through their work on Smarthistory.org (http://smarthistory.org). Maybe our audio-only conversations were less effective than Portland’s videos in bringing this across. These are questions for further research.
6. Indigenous Australia: Object versus thematic guide
Experiences with previous guides strongly influence visitors’ expectations, but the success of the thematic guide, which challenged key assumptions, proved that they do not dictate them entirely. The object guide for Indigenous Australia followed all of the Museum’s usual conventions. Visitors recognised this, citing the number and length of stops, the strong narrator, and the commentaries from experts and Aboriginal artists. They found the guide adequate, but not exceptional.
The audio guide was OK… It had the right number of items and good explanations. It struck a good balance between too many and too few. The length of the comments was good. (IA-S1-21)
Visitors who used the thematic guide were more positive, despite the fact that it challenged their expectations of stop number, length and format. A few had trouble navigating at first, but once they grasped the organising principle, they continued without difficulty. Most visitors immediately understood the approach and could articulate what they liked about it.
When I’ve used other audio guides they’ve been on a specific object, whereas this was on the actual whole section. But I still found it good. I thought on this one it worked quite well the way they did it. And it was a bit different, and I enjoyed that. (IA-S2-12)
Well you know most museums maybe out of the 50 items there’s 25 of them… and people spend most of the time looking for this number on the wall… Here it brought you to an area and all of these objects are described and you get [companion jumps in] into a flow that was much more enjoyable. (IA-S2-19)
The thematic guide satisfied frequent users, even though it may not have met all their expectations. First-timers and occasional guide users were also enthusiastic. Part of its success may be attributable to the desire for a moving narrative; 18 percent exhibition visitors expected this, but it also provided a pattern of movement through the exhibition that visitors appreciated.
7. Where people look
We asked visitors to Indigenous Australia what they focused on while listening. Users of both versions of the guide said they focused on the objects being discussed, but thematic guide users did more looking around. They alternated between specific objects, the rest of the section and simply wandering. The placement of thematic stop numbers on walls and panels encouraged this behaviour, as it was not immediately obvious which object to look at first. Judging by their comments, these varied modes of looking enhanced visitors’ experience.
I liked the way it set the scene and thereafter focused on objects. Other audio guides draw you directly in, but in here first you got the introduction [for each section] and then you can go to objects. I like it better this way, not many are like that. (IA-S2-04)
Because people tended to browse in the vicinity of the stop, the thematic guide worked best in areas of the exhibition that were visually rich. In a sparse area, several visitors got bored. One explained this was because the section did not contain enough material to keep him occupied for the duration of the stop.
The drawing bit [was my least favourite.] There were only two drawings… I wish there was a little bit more there. I wish there’s more things to distract me while I’m still listening to it. (IA-S2-18)
8. Number of stops
Defining beauty had only twelve stops because production costs for conversation were high. Everyone wanted more. By contrast, the object-based guide for Indigenous Australia had fifteen stops, which visitors found acceptable. Many people used the number of stops as a way to reckon value for money, which suggests that communicating this information clearly up front—if it meets or exceeds visitors’ expectations—is a good way to market the guide.
There are no hard and fast rules, however, as reactions to the thematic guide revealed. It had only seven stops, even though the total length of recorded commentary was the same as the object guide. Surprisingly, no one was disappointed. Those who did remark on it were positive: “It compares well, even if it has only 7 stops and other exhibitions have many more stops. But it covered everything” (IA-S2-11).
9. Length of stops
Indigenous Australia visitors who used the object guide liked the two-minute stops which were “just right” and prevented them getting bored or feeling overwhelmed with information. This suggests that the five- to seven-minute thematic stops would have challenged them. Only two of the twenty-two thematic guide users mentioned the length of the stops. They requested more seating so they could rest while listening, but they did not find the length problematic.
10. Guide-driven movement
Across both exhibitions, we investigated three practical aspects of guide usage to better understand how visitors moved through the exhibition: order, repetition and pausing. We discovered that the vast majority of visitors (about 75 percent in Indigenous Australia) listened in sequential order. For those using the thematic guide, this meant they closely paralleled the overarching narrative of the exhibition, which likely contributed to their positive comments about the flow of the guide. Few visitors repeated stops. When they did, it was usually because they had missed something. Only a handful of visitors paused commentaries, even though the audio player had an obvious play/pause button. While interesting, these findings did not explain why audio guide users spent a long time our exhibitions, so we looked deeper.
11. Interpretive ecology
Inspired by Bell’s (2002) description of museum as cultural ecology, our findings led us to consider the exhibition as an interpretive ecology where visitors interact with different modes of information. The different elements of this ecology include text panels and graphics introducing sections of the exhibition, object labels, digital media such as films, the audio guide and the actual object displays. Visitors used complex strategies to navigate this landscape. Many articulated their strategies in surprising detail.
I went through sequentially, I tended to wander while listening and just absorb and then go back and read later on. I used the audio guide. I listened to the introduction in the different sections, and then in the pause I went back and read everything, because I’m a bit of a fanatic for reading all the things—that’s why I’ve been here for three hours. I used that as the basis for directing my attention and then I went back and found all the bits and pieces that they’ve been talking about, so that’s basically how I did it. (IA-S3-22)
Visitors to both exhibitions talked about how they used panels, labels and audio together: 38 percent relied as much on reading as on listening whereas only 10 percent relied on listening alone. For this group, the relationship between written and audio commentaries was extremely important. They felt the purpose of the audio guide was to deliver more information than what appeared in the labels.
You’ve got the written text, so it should provide more documentation, more information about what’s behind the object. (DB-S2-18)
The guide should highlight the important works and add information that is not found in the written descriptions. (DB-S2-26)
I think the audio guide should certainly be there to give extra information than what’s written on your information guide. (IA-S1-02)
These visitors were sensitive to overlap between the labels and audio commentaries. Their value judgements about the quality of the guide were expressed in terms of the amount of repetition between the two. They saw repetition as problematic and it diminished their opinion of the guide.
There was not anything more than I could have read. I want to have a headset to have more than what was written. I didn’t. (IA-S1-04)
It was easy to use, however, it did not tell me anything that I could not read in the boards. It provided some supplementary information but not much more. (IA-S1-09)
Repetition was usually seen as a drawback, but it became an asset when the exhibition was crowded because it was difficult to access the labels.
At times you cannot read, so it is nice to stand back and listen to something, especially when the gallery gets crowded. (DB-S2-14)
Especially if it’s crowded and you can’t get to the display or read it, you tend to listen to the guide. (IA-S2-15)
Groups with specific needs and learning styles also saw repetition as positive:
- Foreign language speakers used it as a way to cross-check that they correctly understood interpretive messages
- Reflective learners appreciated the way both modes of information supported each other
- Auditory learners preferred to use the audio guide exclusively and saw it as a replacement for reading
- Visitors with visual impairments relied on it if reading labels was challenging
Even combined, these categories of visitors were in the minority in our exhibitions, but their desire for repetition presents a contradiction. There is a strong case for why information should be repeated across channels: to ease crowding and provide equal access to all. On the other hand, the majority of guide takers dislike repetition, and their satisfaction depends on avoiding it. Our takeaway is that audio guide commentaries need to be carefully crafted to integrate with the rest of interpretive ecology, especially written labels. Key points from labels should be mentioned, but the bulk of audio commentary should contain additional information. A challenge at our institution is that labels are often finalised very late, long after the audio guide has been recorded.
12. Product strategy
Christensen’s theory of innovation (1997) offers a framework for translating our results into product strategy. He defines two types of innovation: sustaining and disruptive. Sustaining innovations improve on existing products without jeopardising the existing customer base. Disruptive innovations attack an established market from the outside by providing a similar product for a cheaper price. Our experiments provided an example of each:
- The thematic guide in Indigenous Australia is a sustaining innovation. Although it challenged expectations, it gave frequent visitors what they wanted in a format they enjoyed. It delivered on what the team were trying to achieve: a variation on the conventional guide that users were excited about, and, if marketed well, could increase take-up.
- Conversation in the Defining beauty guide shows potential as a disruptive innovation. Occasional and first-time guide users, including the twenty staff members (almost all of whom were under thirty years old and non-guide takers), unanimously liked the conversational format. Despite our improvements in the second version, frequent users still struggled to see its benefits—it just may not be the right product for them.
These findings provide a clear path forward in terms of product strategy. The thematic guide makes sense for exhibitions like Defining beauty, which attract an older audience with intellectual motivations. These visitors have high expectations of what an audio guide should provide. The thematic guide delivered on the most important of these, while still moving the product forward into new territory. Marketing and navigation are challenges for the thematic guide. The number of stops is a key influencer for potential purchasers, and by that measure the thematic guide rates poorly. Instead, messaging should emphasize its strengths: better flow through the exhibition and a coherent narrative. Retail and visitor services staff can provide assistance to visitors who have trouble navigating thematic stops.
For exhibitions that appeal to a younger demographic, a conversational guide could be offered as a mobile app for download on visitors’ own devices. Defining beauty users who liked the conversations drew comparisons with podcasts which suggests that an alternative product that features a variety of voices—especially artists—in dialogue could work for them. Price and positioning are the crucial success factors here. We found that, at £5, on-site exhibition guides are already priced at the top of what visitors are willing to pay. Christensen and Armstrong (1998) explain that new markets embrace disruptive products because they are cheaper than mainstream alternatives. The conversational guide should certainly be priced lower than the on-site guide. It should also be positioned to distinguish it from an audio guide, or it is unlikely to appeal to non-guide takers.
13. Advice for product teams
Some of what we learned can be generalised to other contexts. Here are nine of the most important takeaway messages:
- Challenge conventional audio guide formats, but recognise that you may be risking the satisfaction of your existing user base, which has strong and specific expectations about what an audio guide should be.
- Some strongly expressed expectations are not as important as visitors make them out to be. The thematic guide violated expectations of stop number and length with no serious negative consequences.
- If you are challenging the conventional object-based format, brief public-facing staff on how to help visitors who might be initially confused about navigation.
- Recognise that visitors employ complex strategies in consuming elements of the interpretive ecology. Find out what they are for your specific institution, and design products to support them.
- Investigate visitors’ expectations about how text and audio commentaries work together. Coordinate with other teams to ensure the two complement each other in spite of contradictory production schedules.
- Create predictive models of the kinds of visitors who will attend an exhibition. Had we known Defining beauty was likely to draw a high number of specialists with strong intellectual motivations, we could have pitched our conversations differently.
- If you use a conversational format, justify why contributors have been chosen. State their credentials and area of expertise. Regardless of how eminent or well known they are, explain why their opinion matters.
- Message clearly and prominently about the number of stops and the amount of audio on the guide to help potential customers evaluate its value for money.
- None of the guides we evaluated had layered information or secondary commentaries, but many visitors said they wanted them. Add these content features to boost perceived value for money among frequent guide takers.
The technology for the British Museum’s audio guides has been provided by Korean Air. Defining Beauty: The body in ancient Greek art was sponsored by Julius Baer, with additional support from generous friends of the Museum. Indigenous Australia: Enduring civilisation was supported by BP with the logistics partner IAG Cargo.
Bell, Genevieve. (2002). Making Sense of Museums: The museum as cultural ecology. Intel Labs. June. Consulted January 18, 2016. Available http://echo.iat.sfu.ca/library/bell_02_museum_ecology.pdf
British Museum. (2015a). Spellbound by serenity: A summative report of Defining Beauty: the body in ancient Greek art at the British Museum. Internal document. July.
British Museum. (2015b). Curating conversations: A summative report of Indigenous Australia: enduring civilisation at the British Museum. Internal document. August.
Chang, Ting-Ray. (2015). Evaluation report of Defining beauty multimedia guide: Phase 1. British Museum. Internal document. March.
Christensen, Clayton M. (1997). The Innovator’s Dilemma. Cambridge: Harvard Business Review Press.
Christensen, Clayton M., & Elizabeth Armstrong. (1998). “Disruptive Technologies: A credible thread to leading programs in continuing medical education?” The Journal for Continuing Education in Health Professions 18: 69–80. Available http://www.uic.edu/classes/mhpe/mhpe494dcme/Christensen-Disruptive%20technologies.pdf
Harris, Beth, Christina Olsen, & Steven Zucker. (2010). “Educators, Curators and Docents: Creating Interpretive Resources based on Conversation.” In J. Trant & D. Bearman (eds.). Museums and the Web 2010: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2010. Consulted January 18, 2016. Available http://www.archimuse.com/mw2010/papers/harris/harris.html
Mannion, Shelley, Amalia Sabiescu, & William Robinson. (2015). “An audio state of mind: Understanding behaviour around audio guides and visitor media.” MW2015: Museums and the Web 2015. Published February 1, 2015. Consulted January 18, 2016. Available http://mw2015.museumsandtheweb.com/paper/an-audio-state-of-mind-understanding-behviour-around-audio-guides-and-visitor-media/
Sabiescu, Amalia. (2015a). Defining beauty multimedia guide evaluation: Phase 2 report. British Museum. Internal document. August.
Sabiescu, Amalia. (2015b). Defining beauty multimedia guide evaluation: Comparative report. British Museum. Internal document. November.
Mannion, Shelley, Amalia Sabiescu and William Robinson. "Innovate or stagnate: Disrupting the conventional audio guide." MW2016: Museums and the Web 2016. Published April 2, 2016. Consulted .