What remains invisible? Building infrastructure and collaborative spaces in the Harvard Art Museums

Jeff Steward, Harvard Art Museums, USA, Ming Tu, Harvard Art Museums, USA, Ian Callahan, Harvard Art Museums, USA, Chris Molinski, Harvard Art Museums, USA

Abstract

Keywords: collaborative spaces, experimental platforms, building infrastructure

I. Introduction

From 2009 through 2014, a renovation project led by architect Renzo Piano united the Fogg, Busch-Reisinger, and Arthur M. Sackler museums at Harvard University in an entirely new, expanded facility on the site of the Fogg Museum. The Department of Digital Infrastructure and Emerging Technology (DIET) was tasked with integrating digital technology into the new facility. In collaboration with the Berkman Center for Internet and Society (https://cyber.law.harvard.edu/) and metaLAB (at) Harvard (http://metalab.harvard.edu/), the museums considered several design possibilities for the technical layout and programmatic structure of a tech-heavy space that would support the museums’ mission to “encourage close study of original works of art, enhance access to the collections, support the production of original scholarship, and foster university-wide collaboration across disciplines” (Harvard, 2015).

The museums identified a small room at the top of the new building for a technological intervention.

Figure 1: video wall in the Lightbox Gallery © Nic Lehoux

Figure 2: video wall and projections on the window shades

Originally designed as a conference room, this 500-square-foot space located between the paintings and objects labs of the Straus Center for Conservation and Technical Studies became the Lightbox Gallery. The Lightbox is a multipurpose venue—a space for experiments with technology, collections data, and digital projects in collaboration with campus partners and visiting artists. It is a public venue for research and development.

Managed by DIET and the Division of Academic and Public Programs (DAPP) at the Harvard Art Museums, the Lightbox operates within an elastic mix of curatorial, administrative, and technical demands. This paper outlines the technical and programmatic operation of the Lightbox Gallery from November 2014 through January 2016.

II. The concept

Internal brainstorming

The Lightbox idea emerged from DIET’s attempt to address questions like: How does data fit into the life of an art museum? What can we learn from collections data? How can we experiment with data? What tools can we install in the new building to help us answer these questions? And as a teaching museum on a campus teeming with diverse academic inquiries, can the museums be a nexus for interdisciplinary collaboration?

In May 2012, Jeff Steward, director of DIET, and Jessica Levin Martinez, head of DAPP, tried to answer these questions. After a year of brainstorming, they articulated three core concepts and uses for the Lightbox:

• Collections Access: virtual access to the museums’ collections and collection data
• New Media: a venue for artists working with algorithms, data, physical computing, etc.
• Innovative Classroom: technology as a key tool for cross-disciplinary work

The underlying goal was to assemble the best possible set of modular components so that the environment remains flexible, sparks creativity, and can be reconfigured as projects dictate.

Figure 3: mockup of a large-scale display during R&D

Throughout the brainstorming process, we built prototype projects and demonstrated for the staff the various technology that was being proposed for the space. It continues to be essential to demonstrate and advocate for the value of a technology-driven space in the Harvard Art Museums. These roadshows allow us to test our assumptions about the importance of an environment like the Lightbox in a twenty-first-century museum.

Digital Problem-Solving Initiative

Soon the Lightbox discussion expanded to include campus partners. In 2014, two years after the initial brainstorming sessions, the metaLAB (at) Harvard approached DIET with the idea of pitching the Lightbox as part of the Digital Problem-Solving Initiative (DPSI). Hosted by the Berkman Center for Internet and Society, DPSI is a Harvard-wide project that brings together students, faculty, university officials, fellows, and staff to “work in teams on practicable and concrete digital problems” (DPSI, 2013). The Lightbox was one of five pilot projects tackled by DPSI that year. Led by Jeffrey Schnapp, affiliated professor to the Department of Architecture, professor of Romance Languages and Literature, and director of metaLAB, and Matthew Battles, principal/associate director of metaLAB, the group sketched designs, both structural and programmatic, to jump-start the possibilities for the Lightbox space.

Figure 4: Lightbox Gallery, concept from metaLAB presentation to the Harvard Art Museums, 2014

The group identified several design challenges and asked how those challenges could be turned into opportunities:

• Intimacy: the space is small and interstitial
• Sublimity: its unique character competes with interventions
• Remoteness: not an entryway or waypoint; a destination

In response, the group offered this statement of purpose: “The Lightbox Gallery offers a site for the investigation of the uses of art through media and metadata, enriching the traditional experience of disinterested spectatorship” (metaLAB, 2014). The Lightbox could become a “platform for combing through connections, juxtapositions, and family relations, posing new meanings and uses for the collection” (metaLAB, 2014). It could be used to explore:

• Infrastructure: a laboratory for interactive, multisensory media
• Interpretation: a vantage point for seeing the collection in its entirety
• Creation: a studio and gallery for making and sharing exciting new work
• Activation: channels for dialogue within museum spaces and beyond

As Battles wrote in the proposal: “Through media and metadata, the trapdoor connecting an object’s surface to its depths may be sprung open. The Lightbox Gallery is a materials lab for playing with use, form, meaning, value—for discovering the ways in which objects in collections speak to one another, and to us” (metaLAB, 2014).

Zeroing in

The collaboration with DPSI helped confirm the soundness of the original concepts for the space. In summer 2014, we started to specify the hardware that would take this imagined space and make it real.

Figure 5: time lapse of construction

III. The infrastructure

The Lightbox Gallery hardware consists of these main components: a 3-screen-by-3-screen LCD-based video wall, two DLP projectors, a sound system, a video processor, an AV control processor, a DigitalMedia switcher, two Windows 7–based media servers, and a variety of video input interfaces. AdTech Systems (http://www.adtechsystems.com/) completed final system design and installation. What follows is a description of the system infrastructure, the rationale behind that infrastructure, and some of the challenges we faced in putting it to use.

Figure 6: system diagram (solid lines represent video signals)

The displays

If budgetary or time constraints eliminate the possibility of commissioning a custom LCD fabrication, there are three possible alternatives for building a video wall: commercial LCD panels, projection, or a modular video wall product such as Christie MicroTiles. Each has their own strengths and weaknesses.

Table 1: video wall options

Each solution has its advantages depending on your situation. We chose an LCD panel system because we prioritized pixel density, installation depth, and noise reduction. Because the Lightbox Gallery sits on the top floor just beneath the glass roof, bright-light image performance was also a high priority. MicroTiles were a strong contender because of their color performance in bright light. However, the installation depth required (nearly a foot) would have taken up far too much space in the 500-square-foot gallery, and the cost was prohibitive.

Behind the screen

While the display is the single most important set of components in a video wall, the hardware and software delivering images to that screen have a huge impact on image quality, usability, and flexibility. Because we wanted the displays to accept many different input types and be flexible with image arrangements, we had to deploy a video processor. A video processor is a piece of hardware that is able to treat all of the available pixels as a single canvas. In our case, the video processor is two connected Vista Spyder X20s; our canvas is nine LCD panels and two DLP projectors. The arrangement of image sources on that canvas can then be manipulated either programmatically or through a real-time WYSIWYG interface running on a PC with a network connection to the processor. Our processor is programmed to provide image arrangements suitable for a variety of use-cases. These include doing multisource presentations, scaling a single source to the full screen, and our default configuration of a single rack mounted source with a 5760×3240 desktop. (In addition, the video processor is able to compensate for some of the physical features of our space. The processor is “aware” of the structural columns between the window shades that we use as a projection surface. It blacks out those areas so that we are not projecting onto the columns.)

Figure 7: programmed image configurations for video wall

Figure 8: video processor adjusting the image to compensate for the structural columns

Media servers and pixel bottleneck

One of the benefits of using a video processor is that it creates a layer of abstraction between your sources and your display. Want to scale a single 1920×1080 source across your 5760×3240 display? Not a problem with a high-quality processor. For scaling 16:9 video and presentations, this is a fantastic feature. Our AV installer, AdTech, initially designed the system with these sorts of business-oriented use-cases in mind. However, as originally spec’d by AdTech, the signal-switching hardware and input capacity of the video processor would have created a pixel bottleneck. In other words, the display resolution of the system would have been greater than the input capacity. That would have meant that all image sources would need to be scaled up.

We quickly realized that such a setup would not suit our needs. As a museum, we are heavily invested in aesthetics and require control over every pixel. We did not want the video processor to interpolate pixels that were not there. Although it was less cost effective than AdTech’s original configuration, we doubled the switching and video processor capacity so that the system would be capable of taking an image source for every physical pixel available on the displays. This, however, introduced another set of problems: what hardware has the capacity to output resolutions greater than 4K?

If we want to display data visualizations, photos, and other content at maximum resolution, we need a PC capable of doing so—a custom-built machine is our only option. We configured a media server that uses two workstation video cards (AMD FirePro W9100) with a total of twelve digital outputs. This means that we have a discrete output on our main image source for each of the eleven physical displays in the gallery.

With Windows 7 running on the machine, we were able to get a desktop that spanned the entirety of our display. We could open a high-resolution conservation photograph and expand it across the entire video wall. But there was a catch. If we tried to use the full-screen option or clicked the maximize button on a window, it would only use our primary display (i.e., the first output from the first video card). Things got better when we configured the AMD Eyefinity driver to stitch the outputs together as a single display.

Unfortunately, the Eyefinity software only works with outputs that are on the same physical card. With eleven outputs spread across two six-output cards, that still left us with two virtual displays: a 3×2 and a 5×1. In trying to work with such high resolutions in a desktop environment, we were bumping up against the limitations of the operating system. It eventually became clear that we would have to use custom software in order to show true full-screen content.

Figure 9: Windows display dialog

This is not difficult to do with Processing (http://processing.org). But it does limit our ability to use commercially available applications to go full screen. To further mitigate this issue, we decided to configure a separate media server for the two projectors. We now have two Windows machines. The first machine is configured as if it has two displays that map to the top 3×2 and bottom 3×1 portions of the video wall. The second machine is configured as a single 2×1 display for the projectors.

Everything-over-Ethernet

The Lightbox Gallery is on the fifth floor. The equipment rack for driving the system is underground on the Lower Level. So all of the signals that connect the source equipment with the gallery must travel about three hundred feet through the building. Wireless transmission is not an option for any of our signals.

Instead, the system relies on more than twenty long runs of shielded CAT-6 cable. These cables provide video transmission (using DVI-over-Ethernet converters for the displays and DM for the in-gallery HDMI and VGA jacks), a private AV LAN for control, and USB connections to the media servers via USB-over-Ethernet adapters.

Figure 10: lots and lots of cable

During the installation, we requested that extra cabling be run between the gallery and our equipment room, as well as inside the equipment room between the in-room rack and the floor boxes. These undedicated cables will allow us to expand the capabilities of the system at some point in the future, such as using the cabling in one of the floor boxes to connect a touchscreen to a remote PC.

Touchscreen interface

The only piece of infrastructure that we have not yet discussed is a touch panel located on the wall inside the gallery. This passcode-protected touch panel communicates with the control processor to select content sources and map them to the selected display configuration. It is possible, for instance, to select the HDMI wall jack and map it to the lower right-hand corner of the video wall while displaying content from the media server PC on the upper left-hand corner of the video wall. The touch screen is also where staff programs the devices’ on/off schedules, to accommodate after-hours programs, for example.

Figure 11: the in-gallery control touch panel

Management and staffing

DIET and DAPP jointly manage the Lightbox. DIET provides oversight and support of the equipment and conducts research. DAPP handles the programmatic aspect and scheduling of the space and does outreach to engage collaborators and campus partners.

The Lightbox infrastructure is not designed for one fixed project. Its flexibility allows the digital platform to always be active and relevant for visitors, museum staff and professionals, educators, and its entire community. With a few mouse clicks, a collaborator can connect his or her laptop to the display system and control the entire video wall in the Lightbox. With only one key press, a collaborator can switch between different digital projects in the gallery.

Supporting this degree of flexibility requires in-house professional personnel to constantly develop engaging programs and creative digital installations. We have found no turnkey commercial solutions that are adequate for running an environment like the Lightbox Gallery.

Figure 12: timeline showing the planning, construction, and events during the first year of operation

In the end, the responsibility for sustaining technology-based galleries such as the Lightbox rests with humans, not with the technology. Four full-time staff form the core Lightbox project development team: a project manager who oversees the program (DIET director); a program developer who seeks collaborators, develops relevant programs, and promotes the projects (DAPP curatorial fellow); an audio/visual technology specialist who maintains the technology infrastructure (DIET technology support specialist); and a user interface/user experience designer and developer who works closely with collaborators to develop interpretive digital content and design interactive experiences for visitors (DIET technology fellow).

This organization and staffing assures that the gallery is not just a blank canvas, but a space that enables innovative visitor experiences.

IV. Program development and projects

To date, the Lightbox has hosted artists, museum conservators, an archaeological expedition team, museum archivists, astrophysicists, digital humanities groups, students and faculty across the campus, and many others.

During the first year of operation, we developed thirty different projects and facilitated a series of diverse events that utilized the digital display walls, audio system, and input/output systems that connect with the entire museum building. The projects and programs ranged from interactive multimedia artworks, data visualization of the museums’ collections, and conservation studies to community science festivals and class presentations.

Video 1: Lightbox Gallery Projects (https://vimeo.com/151150924)

Video 2: Lightbox Gallery Teaser (https://vimeo.com/151161408)

Not many art museums in the United States operate a digital technology-based gallery with this open-ended setting and experimental spirit. DIET is constantly testing new frameworks and tools to answer the question of how museums can rapidly and continuously develop content-rich and engaging programs for the gallery while keeping technologies financially sustainable.

Collaboration in a cross-disciplinary and open-ended platform

As a multipurpose venue, the Lightbox not only serves as an exhibition gallery, but also acts as a workshop, studio, meeting room, and interactive lab. With technology this open ended, close collaboration is necessary for the success of any proposed project or program.

Project kickoff meetings are held in the Lightbox Gallery, as are almost ninety percent of meetings throughout the collaboration process. As the core Lightbox team introduces the gallery and demonstrates sample projects, the immersive experience of being in the venue becomes apparent and collaborators begin to explore the Lightbox’s possibilities.

DIET also developed a Web-based, 3D, virtual Lightbox Gallery (http://apps.harvardartmuseums.org/lightbox/) that serves as an interactive tool for demonstrating featured Lightbox projects and allows collaborators to explore the possibilities on their own.

Figure 13: a Web-based, interactive, virtual Lightbox for remote testing, collaboration, and demos

The Lightbox team then works with collaborators to transform their ideas into something tangible. Feedback from collaborators has shown that discussing ideas in the space and seeing its non-traditional tools in action helped them expand their vision for telling their story.

Figure 14: a storyboard created during project development

We do not close the gallery to visitors during the iterative prototyping phase, nor at any other point during project development. The visible conservation labs next to the Lightbox gallery are part of the museums’ desire to be transparent about the work we do. The Lightbox gallery is able to take that transparency a step further. When visitors walk into the Lightbox during testing, they’re invited to experiment with prototypes, and their feedback is often incorporated into future iterations. Harvard Art Museums emphasize hand-on experiences and audience engagement; the Lightbox gives audiences a sense of ownership in these collaborative projects.

Figure 15: collaborators reviewing and editing a project during development

Before the official launch of a project, collaborators and the Lightbox team schedule related in-gallery talks and events. In-gallery talks give visitors the opportunity to learn more about the project directly from the creators. At the same time, the creators are able to learn directly from visitors what works and what doesn’t. In addition to this qualitative feedback, the Lightbox core team utilizes Google Analytics to gather quantitative data about user interaction. Real-time data collected by these analytic tools allow the Lightbox team to immediately respond to and fix issues.

Figure 16: an audience during an in-gallery event

Challenge 1: Don’t be afraid of the blank slate

The first challenge of developing a Lightbox project is the difficulty of transferring an idea to an installation in an open-ended gallery. When a collaborator visits the Lightbox for the first time, the venue usually impresses by its great view and powerful infrastructure. The possibilities can be overwhelming. Faced with a blank slate, where do you begin? This multipurpose, open-ended, technology-laden setting is not what people typically imagine when they think about gallery spaces in art museums.

Imagine the first-generation iPhone. Its powerful technologies allowed users to download or build previously unimaginable applications to extend its functionality. However, it took a long time for people to see it as anything other than a small computer screen. As more applications were created, developers and users began to realize the open-ended platform indeed generates innovative user experiences and has unlimited possibilities.

The Lightbox is open ended the way a blank canvas is open ended, which can be daunting. To boost collaborators’ imaginations to create engaging and relevant applications, the Lightbox team developed various sample projects as inspiration. Also, the team has created fourteen templates that allow collaborators to view their content in the Lightbox in as little as thirty minutes.

Furthermore, the Lightbox team continues to do research and development on technologies including APIs, computer vision, physical computing gadgets, remote controls, electronic paper, bone conduction speakers, and interactive 3D printed paintings.

Challenge 2: Think beyond the screen

After learning that the video wall is powered by a Windows computer, it is hard for collaborators to think beyond the idea of the video wall as a large, high-resolution computer screen. The interaction between visitors and the digital platform does not happen at arm’s length. Collaborators need to remember to design for the whole space, not just the screen.

Currently most of the Lightbox projects involve customized user interfaces along with different remote controls for interacting with the digital content. Visitors can have a close look at the visualization in the gallery or choose to view at a distance. They can view projects on the screen or consider the abstractions created by reflections on the glass walls of the space. Offering multiple ways to perceive and interact with digital installations lets visitors create their own insights.

Figure 17: the holographic effect of the video wall reflected in the glass walls of the Lightbox Gallery

Challenge 3: Designing user experiences

Over the past few years, interpretive technologies in gallery spaces have increasingly focused on digital displays, and touch screens in particular. By limiting our thinking to two-dimensional screens with touch as the only user interaction, museums neglect the fact that we offer multisensory experiences, not merely visual.

“We live not in the digital, not in the physical, but in the kind of minestrone that our mind makes of the two,” says Paola Antonelli (2015), the senior curator of the department of architecture and design, as well as the director of R&D at the Museum of Modern Art in New York, in a recent salon at the Harvard University Graduate School of Design. The rapid development of digital technologies over past decades has dramatically changed people’s lives and the way humans perceive this world. People in the twenty-first century inhabit and learn in a hybrid world, one that now blurs lines between the physical world and digital experiences such as websites, Web/mobile applications, and the Internet of Things.

Figure 18: customizing an off-the-shelf remote control for interacting with Lightbox projects

Physical computing can create more engaging and meaningful museum experiences than those offered by traditional digital display strategies.

Touch provides visitors a simple and familiar way to interact with digital platforms in physical galleries. However, restricting interactions to touch limits the imagination and possible interpretations of art. The interaction itself should be considered and used as a way to interpret the digital content and, most important, make meaning for the users.

Figure 19: experimenting with controllers

V. Conclusion

It is not surprising that people find it difficult to understand this new concept for a gallery. For most institutions, this degree of open-ended and rapidly changing content development would result in a space that is unmanageable and understaffed. It therefore makes sense that digital platforms in most art museums and galleries are built for a single purpose, a permanent digital installation, or a temporary exhibition. As a teaching museum, it is our responsibility to push the boundaries of what a gallery can offer.

Our most significant barriers have been collaborators’ varied understanding of the gallery’s capabilities and the lack of concrete examples. So far, only a quarter of the technology features in the Lightbox have been used. Our ongoing challenge is to broaden collaborators’ imaginations and conceptions of what this space can be.

The Lightbox Gallery is much more than a cutting-edge, gallery-sized theater. It is a new model for interpretive museum technology in the twenty-first century.

Figure 20: experimental data visualization and hacked Ikea alarm clock controller

References

Antonelli, Paola. (2015). “Hybrid: The Space in Between,” salon at Harvard University Graduate School of Design. Available http://www.gsd.harvard.edu/#/events/hybrid-the-space-in-between.html

DPSI. (2013). “About DPSI.” Available https://cyber.law.harvard.edu/research/dpsi

Harvard Art Museums. (2015). “Mission.” Available http://www.harvardartmuseums.org/about/mission

metaLAB. (2014). “Presentation to the Harvard Art Museum.” metaLAB (at) Harvard.