Moving from downloads to uploads: Toward an understanding of the curricular implications of access to large scale digitized museum collections on the professional practice of K–12 classroom educators

Darren Milligan, Smithsonian Institution, USA


The need for museum strategy to be audience driven is now directed and enabled (in an accelerated way) by digital technologies. This allows, or requires, museums to understand the intersection between the needs of those it hopes to serve and the capacity of its own organization to meet them: to provide customized experiences and opportunities to unique audience groups. Educators are one of museums' historically most-valued audiences. Opportunities to have greater impact with teachers, their students, and the learning experiences they create, are great—greater than in the past, when museums focused on adult programming, school visitation, and exhibition-centered lesson plans. These new opportunities lie primarily in the utilization of museum collections and resources within the classroom, where the teacher can make use of them in ways that fit naturally into the learning process they have already developed for their students. To enable this, as we should, museums need to understand this group and how they use digital assets to design and deliver learning experiences. This study looks towards the development of a reusable framework for addressing this need through an understanding of the evolving role of the museum in the education space, the process and knowledge bases required for teachers to be designers of learning resources, and finally the ways that technology itself (in this case, primarily the Web) changes the nature of teaching and learning. The framework proposed is used to develop a survey instrument that is then tested through a case study of an emergent digital platform for teachers, the Smithsonian Learning Lab.

Keywords: teachers, education, design, technology, pedagogy

1. Introduction

An efficient educational museum may be described as a collection of instructive labels, each illustrated by a well-selected specimen. (Goode, 1891, p.433)

While at first museums may have been hesitant about their place on the Internet, as Howes (2007) describes, their goals and the opportunities afforded by these technologies are, in fact, complementary to our own: to increase access to information and knowledge. Beginning in the 1990s, with paid access through projects like the Bridgeman Art Library to publishing on popular platforms like the Flickr Commons and the Google Cultural Institute, museums have more and more recognized the value to the public, to the preservation of their collections, and to the longevity of their institutions, of broad digital distribution of their collections and metadata (Sanderhoff, 2014).

Museums can make meaningful contributions to societies structured upon the use of information (Gurian, 1999). This role is one that, since the 1960s, citizens have begun to expect from their cultural and scientific institutions. Williams describes the change within cultural heritage in this way: “Once the quiet, undisturbed sanctuary of scholars and researchers, museums were seen now as public trusts with duties and responsibilities to their collections, to their communities, and to future generations” (2010, p. 16). Within this context, museums can see themselves as producers of information, as the “primary resources, or commodity, of the museum business as information, rather than as artifacts” (p. 74). Or even more broadly, museums are no longer just in the “salvage and warehouse business” (Franco, as quoted in Weil, 2007, p. 30), but have, through the utilization of information and increased access become a technology itself, a “highly adaptable instrument that can be employed for a wide range of purposes” (Weil, 2007, p. 31).

The speed, diversity, and public excitement attached to this new utility, facilitated through museum collection digitization, have grown exponentially in the past decade (Heyman, 2015; Lohr, 2014; Roberts, 2014; Siegal, 2013). As these assets become near universally available to a geographically dispersed population, it is essential for both effectiveness and sustainability that the institutions leading this change understand the practical implications to intended audiences. One of those audiences, or perhaps we should call them users, comprises individuals in educational settings. Museums have taken on educational missions since their establishment in the modern era, creating programs and resources for both adult and child learning (Silver, 1978). We see this tradition continue today in the manner in which museums explain their own role, in their mission statements, in which many major international institutions either use the word “education” directly or imply it in describing what they hope to achieve.

  • Metropolitan Museum of Art: “collects, studies, conserves, and presents significant works of artin order to connect people to creativity, knowledge, and ideas” (
  • Museo Nacional del Prado: “Impulsar el conocimiento y difusión de las obras y de la identidad del patrimonio histórico adscrito al Museo, favoreciendo el desarrollo de programas de educación y actividades de divulgación cultural.” (
  • Tate: “To promote public understanding and enjoyment of British, modern and contemporary art.” (
  • Smithsonian Institution: “The increase and diffusion of knowledge(

2. Aim and methodology

This paper will explore the changing role of museums in light of their impact on educators, specifically on K–12 classroom teachers. As museums digitize and share their collections at unprecedented scale (augmenting their focus from the local museum visitor to the national or international user of digitally delivered content), it is incumbent upon them to develop a practical understanding of the impact of these activities. An academic approach will be taken to understand and analyze frameworks from three fields, with the goal of developing a reusable survey instrument to be utilized in future research. The instrument will be tested with a limited focus group of educators. The three areas to be explored are:

  1. The modern role of the museum as an educational institution
  2. Classroom teachers and their specific behaviors as instructional designers
  3. Frameworks to evaluate the impacts of technological integration on these behaviors

We will examine each of these components individually in the sections that follow: first within the literature; then through the lens of a case study, the Smithsonian Learning Lab; and finally by survey data collected from prospective users of this project.

3. The modern museum as educational institution

So far it has been assumed that the students will go to the museum. This is to a great extent desirable, for they should become familiar with the interior of this building as early as possible, but some of the material would be of more practical use if it could be handled in the school class room. (Farnum, 1919, p. 195)

Silver (1978) provides a substantial history of the nineteenth- and early twentieth-century social movements that have incentivized and evolved the museum’s role as educational platform. Museums have taken on some educational activities, as evidenced through programming, since their establishment in the modern era. Many examples exist of the development of both programming for adults (the Metropolitan Museum of Art’s adult lecture series began in 1872 and the Boston Museum of Fine Arts adult classes in 1876) (Silver, 1978) and for children (in 1901, the Smithsonian converted a portion of its limited space into the “Children’s Room,” one of the first museums to develop exhibition space specifically designed for the needs of young people (Smithsonian Institution Archives, 2009)). Later examples exist of museums providing resources directly to classrooms. As early as 1910, the Philadelphia Commercial Museum loaned extensively to Pennsylvania public schools (figure 1) both informational materials (lecture slides and scripts) and collection objects (primarily geological specimens).

… the materials in the lending collections are functioning, educationally, to better advantage than do many of the class trips to museums simply because the teacher receives her material when she needs it and when it fits into the work being done. (Peters, as quoted in Wolfrom, 2010)


Figure 1: photograph of loan objects (consisting of support materials for public school lectures) from the Philadelphia Commercial Museum, 1910. (used under a CC BY-NC-ND 2.0 license;, retrieved July 28, 2015).

The late twentieth-century museum’s focus on addressing public expectations and needs soon converged into a twenty-first-century interest in how specifically to serve them. This period, now referred to as “new museology,” was driven by three primary factors: 1) economic changes that forced institutions to appeal more to the needs of their visitors, 2) rise of educators within the leadership roles of museum professional organizations, and 3) the founding of new museums whose financial limitations prevented them from amassing collections that could compete with older institutions (Weil, 2007).

With this demand, a growing field of museum education grew to better understand how people learn within museums, as well as while using museum objects (interested readers should seek out work by Rennie and Johnston (2004) and Falk and Dierking’s Contextual Model of Learning (2000, 2012)). Object-based learning theory has its foundations in the curatorial practice of object study and exhibition in the nineteenth century, but grew in popularity primarily in the twentieth century with the convergence of a growing interest in the anthropological investigation of objects; the growing value of material culture in the understanding of humans; the study of museum visitor behavior by psychological researchers; and the growth of early childhood educators who emphasized hands-on learning, play, and the direct study of objects (Paris, 2002). The literature points to this pedagogical approach as having many benefits, including:

  • Linkage of student interest to the construction of knowledge (Ash & Wells, 2006; Chatterjee, 2007; Duhs, n.d.; Eberbach & Crowley, 2005; Sotto, 2012)
  • Utilization of all senses, with a specific focus on touch, sight, and smell (of which touch and smell are rarely included in traditional classroom learning experiences) (Duhs, n.d.; Leinhardt, Crowley, & Knutson, 2002)
  • Ideal for group and participatory learning pedagogies (Duhs, n.d.)
  • Facilitation of the acquisition of skills in observation as well as drawing (Chatterjee, 2007; Rice & Yenawine, 2002)

Many museums now receive more digital or online visitors than in person, driven primarily by the growth of digitization. This shift in access has opened new areas of investigation. Learning through digital surrogates offers specific opportunities that museums, which have physical limitations, do not. Frost (2002), among others, explores both sides of this explanation, as digital exposure to the arts is often the principal exposure for many, and therefore it is crucial to understand that the surrogate offers a different experience than the physical. Among the benefits of learning through objects in the digital space are:

  • Access to large numbers of objects
  • Opportunities for rich linked contextual information about the objects and improved discoverability
  • Opportunities for two-way collaboration and contribution facilitated by upload and mashup (Frost, 2002; Lindquist & Long, 2011)

The limitations of digitally enabled object-based learning experiences center on the very limitations of two-dimensional representations of often three-dimensional objects: the loss of sensory-based understanding of scale, color, and quality (Frost, 2002). Despite these limitations, few would deny that digital access to museum collections plays a substantial role in serving schools and the learners. This increase in access, combined with the continued decline of the museum field trip (Ellerson & McCord, 2009) and the disconnect between how museum lesson plans are designed and used (Milligan & Wadman, 2015), leads us to consider how best to reshape the role that museums play in serving teachers and learners. Several recent studies conducted by museums and libraries have attempted to understand how teachers specifically make use of digital museum resources: both digitized collections and other prescribed learning resources.

Research on teacher use of digital museum resources

From 2008 to 2015, four museum-based research projects focused on teachers’ needs and behavior with digital collections.

1. Council of Australasian Museum Directors: Museum and education digital content exchange

From 2008 to 2009, three Australian museums conducted a study to gauge the impact of their digital learning resources and address what they saw as a potentially unnecessary siloing of educational content on museum websites (Baker, 2009; Chan, 2009). To make collection content more useful within the classroom, they recommend the following:

  • Adopt a standardized way of describing digital resources (via metadata), based upon a museum standard, but with the addition of educationally relevant fields
  • Provide open and consistent licenses for educational purposes
  • Encourage collaborative online environments that enable customization
  • Offer support and models of effective educational use of museum digital collections

2. Teacher programs: Assessing the Getty Museum’s online resources for K–12 teachers

To improve online resources for its teacher audiences, the Getty Museum conducted an online survey to better understand how teachers use technology in their own classrooms, and specifically how they teach using the collections of the museum (specifically, works of art) (Sotto, 2012). Recommendations include:

  • Put lesson plans in downloadable and customizable formats (i.e., match format to actual classroom usage data: e.g., PowerPoint for high school teachers, illustrated workbooks for lower grades)
  • Expand target audiences to include PreK–3 and improve high school resources
  • Create new lessons in topics closely aligned to teacher needs, including those with a cross-disciplinary focus (e.g., using art to teach math)

Recommendations for the artwork content include:

  • Add information on the artist, time periods, and genres in which the work was created
  • Include information written with teachers in mind
  • Provide flexible image viewing opportunities, including for download at print size

3. Digital Public Library of America: “Using large digital collections in education: Meeting the needs of teachers and students”

The Digital Public Library of America study is the first to look specifically at large and complex digital collections with the intent of providing organizational direction for addressing educational target audiences’ needs (Abbott & Cohen, 2015). The study focuses on how to present digital primary sources (as well as tools for teachers to curate their own sets of primary sources) and recommends the following:

  • Develop curated primary source sets to serve the needs of K–12 teachers and students
  • Build a network of teachers to create and review sets
  • Offer users tools to curate their own sets
  • Support a community of practice through education outreach, regular meetings, and networking opportunities

4. Smithsonian Digital Learning Resources Project

My own team at the Smithsonian Center for Learning and Digital Access built upon these previous studies. From 2012 to 2015, we conducted a series of studies to better understand the educational uses of our digital resources and provide a roadmap for future digital development (Milligan & Wadman, 2015). The specific research objectives focused on understanding classroom teachers’ ability to identify, analyze, and extract digital content, with the ultimate goal of enabling all users to achieve their own personal learning objectives through the Smithsonian’s resources (i.e., to foster online users who are active creators of digital resources personalized for learning in their own classroom). Its recommendations include:

  • Improve search with tools like autocomplete and/or spelling assist
  • Provide scannable grade-level and subject information (to quickly analyze usefulness)
  • Offer resources from a wide variety of sources
  • Publish educational resources that are interdisciplinary and/or multidisciplinary, connected to students’ interests, aligned to teaching standards, adaptable, and downloadable
  • Build platforms that contain tools for student interaction with the resources, a wide variety of sharing options, and the ability to save and structure resources for later review/use


These studies’ primary focus was on the teachers’ use of museum-created learning resources, not per se on digital collections. In other words, the emphasis has been on consumption (“downloads”). And while all worked with teachers directly to understand their needs, they lack the critical understanding of teaching practice, especially when it comes to understanding how teachers behave when they use or integrate digital resources in their lessons—that is, how teachers behave as designers (“uploads”).

4. Teachers as instructional designers

With a commitment to research and evaluation as the basis of our work, the Smithsonian found it important to ground this investigation in the work of leading educational design theorists. What follows is a summation of research results and theories that suggest foundations for decision making in ensuring that we meet the needs of educational users.

Teachers in pre-service training are rarely taught the discipline of design (and if they are, the instruction is often out of sync with actual practice), focusing rather on basic lesson planning (McKenney et al., 2015). One might not be surprised that the literature points to a less than systematic approach to the design of instructional materials by teachers (Kerr, 1981). Rather than beginning with more objective strategies, teachers typically begin with the materials at hand (Taylor, 1980; Yinger, 1980) and consider the context in which they need to teach (Taylor, 1970; Voogt et al., 2013), including the individual circumstances of their students, their classroom, and their school (expectations of external stakeholders, including principals, colleagues, and even the community), but also their prior knowledge and beliefs (Boschman, McKenney, & Voogt, 2014; Voogt et al., 2013). The practical typically outweighs the potential for a more systematic approach (Kerr, 1981). They then develop an activity, or adapt an existing one, and only lastly focus specifically on the outcomes or learning goals they hope to see from their students (Clark & Yinger, 1980; Taylor, 1970; Taylor, 1980). It is the students, their needs, and an activity’s potential to engage them that receive special emphasis (Boschman, McKenney, & Voogt, 2014; Davis et al., 2011; Jackson, 1966, 1968).

An indication of the importance of the design-centric twenty-first century classroom can be be found in the teaching standards of the International Society for Technology in Education, a leading not-for-profit organization that supports educators in their integration of information technologies into teaching:

Design and develop digital age learning experiences and assessments: Teachers design, develop, and evaluate authentic learning experiences and assessments incorporating contemporary tools and resources to maximize content learning in context and to develop the knowledge, skills, and attitudes identified in the Standards (ISTE, 2008)

In terms of the integration of technology (the focus of this paper: museum digital assets and the systems that deliver them), Boschman, McKenney, and Voogt (2014) see that a teacher’s direct involvement in the design of technology-enhanced learning is crucial in various aspects. It sustains usage and innovation, increases ownership, provides opportunities for technological skill development, and results in the development of materials better suited to the needs of a particular classroom. However, the mere integration of technology does not necessarily fundamentally change the generalized competencies of effective instructional design and the steps used to develop useful digital resources. However, the knowledge, skills, and attitudes required for the effective integration of technology do change (Kirschner, 2015). So, how are teachers using digital technology?

Teachers’ use of technology in the classroom

In 2012, the Pew Research Center’s Internet & American Life Project conducted one of the first large-scale studies of US teachers and their use of digital technologies (Purcell et al., 2013). The study illuminates how technology is being used in the classroom and how these technologies (both hardware and the Internet) have changed the nature of education and the knowledge needed to do work in this environment.

Teachers are integrating technology into their instruction. Computers and projectors (nearly ubiquitous), along with mobile devices and digital cameras are the most frequently reported hardware being utilized. Seventy-three percent of survey respondents report that mobile devices are used regularly in teaching and are relied on for the completion of assignments. An almost universal 88 percent of American teens have access to a mobile phone, while 73 percent have access to a smartphone (Lenhart, 2015).

Teachers too are using the Internet directly with students, for research (95 percent), accessing (79 percent), and submitting (76 percent) assignments. However, more participatory activities like online discussions, collaborative assignments, co-editing, and providing publicly accessible work samples remain still emergent.

Understanding the impact of the rise of technology and its usage in the classroom is relevant. Seventy-five percent of survey respondents report that “increasing the range of content and skills you need to be knowledgeable about” has a major impact on them personally, and 82 percent have sought opportunities to learn more about how to incorporate digital technologies into the learning process, as most rely commonly on their own research and experience when developing new resources for use in the classroom (as compared to developing skills in formal training).

Teachers as designers of technology-enhanced learning

A field of study has developed to better understand how teachers act in this area of instructional design. Teachers engage in the design of technology-enhanced learning, as it affords ample opportunities for resource personalization (to match the often individual needs of particular students) and wide flexibility to engage and evolve curriculum, and allows resources to be practically implemented (Kali, McKenney, & Sagy, 2015). To better understand this process, McKenney et al. (2015) analyzed research in classic design fields (graphic design, architecture, etc.) and instructional design (non-technology-enhanced) to understand how the roles that teachers play in designing technology-rich learning activities can be understood within the literature. Cvico, McKenney, and Voogt (2014) described three roles:

  • Enactors: As enactors, teachers discover and use ready-made or ready-to-teach resources, making small adjustments only after students have responded.
  • Re-designers: In the role of a re-designer, a teacher pre-adapts existing materials beyond simple reactive tweaks, in order to integrate them into their curriculum.
  • Co-designers: As a co-designer, teachers work collaboratively to develop completely novel learning activities, preferring (at times) to work alone or in small groups to design short activities and lessons, and in large teams to create cross-disciplinary modules.

The roles vary in their levels of customization: the making of small adjustments to tailor or adapt learning materials to a particular situation, setting, or individual. While these changes are sometimes directed by the evidence garnered from student work, as Matuk, Linn, and Eylon (2015) point out, it is more often driven by practical concerns and mere feasibility (to adapt a lesson to fit a shorter class period, for example).

To conceptualize these behaviors, McKenney et al. (2015, p. 190) developed an “Ecological Framework” that looks at the six aspects of design knowledge required for teachers to be effective reflective practitioners. The framework combines an awareness of the knowledge required for teachers to fulfill their roles as designers, as well as their confidence in implementing technology-enhanced learning opportunities.

  1. What fundamental knowledge is required
  2. Why certain things should be developed and implemented
  3. How to develop and implement them
  4. When to act or implement tools and techniques
  5. Whom to consult for optimal design and implementation
  6. Where the design should be implemented

The integration of technology in education is often described as the seed for substantial evolution of teaching and also learning; however, this change has often lagged behind the transformation seen in other fields. The focus has often remained on training teachers on what technologies to use rather than on how to integrate their usage into practice. A theoretical approach is needed to best understand and support this complex integration.

5. The integration of technology, pedagogy, and content knowledge

Technological Pedagogical Content Knowledge framework

Historically, the focus of teacher training was not on these design knowledge bases, but rather on content knowledge: the knowledge that teachers were passing on to students. More recently, with the growth of information technologies, the emphasis has shifted to generalized pedagogical approaches. But, as Laurillard (2010, p. 3) summarizes, “Knowledge technologies shape what is learned by changing how it is learned,” so it is imperative to address these changes in an integrative way. To do this, Shulman (1986) introduced the Pedagogical Content Knowledge Model to integrate and address the interdependencies of these two areas. What is novel in his approach is that it demands more than mere content and pedagogical mastery; it requires the “blending of content and pedagogy into an understanding of how particular aspects of subject matter are organized, adapted, and represented for instruction” (Mishra & Koehler, 2006, p. 1021). In other words, the pedagogy should change and adapt to the specific needs of the content at hand.

Shulman did not discuss technology (digital or other) specifically in his model, as he was developing it right before widespread consumer computer adoption. As Mishra and Koehler explain, technology, specifically computers (both hardware and software) and the experiences they enable via applications and the Internet, since the 1980s, have moved out of a “transparent” stage (one in which non-digital technologies, such as books, are so ubiquitous as well as stable in format and function that they, through widespread adoption, are minimized to the background of instructional strategy) and have “come to the forefront of educational discourse primarily because of the availability of a range of new, primarily digital, technologies and requirements for learning how to apply to teaching” (2006, p. 1023). Digital technologies have substituted for other non-digital mediums, but also, through their consistent evolution, presented a new challenge to adoption in an era previously defined by stability. In this past, teachers and the students they educated focused primarily on the relative content and the approaches to teaching it (pedagogy), but in these new realities, knowledge of the integration of technology into these two previous contexts appears crucial. For the purposes of this paper, the Technological Pedagogical Content Knowledge Framework (TPCK, or later called TPACK, for ease of pronunciation), developed through focused research by Punya Mishra and Matthew Koehler at Michigan State University, best fits our needs (2006).


Figure 2: Technological Pedagogical Content Knowledge (TPACK) framework. Reproduced by permission of the publisher, © 2012 by

What is unique about the TPACK framework is that it considers the intersection of the three knowledge bases in two ways. First, the individual knowledge bases are considered at the core knowledge base level, and then the intersections of them, first pairs and then as a triple, are examined (Pamuk et al., 2013) to create seven knowledge bases.

Technological Pedagogical Content Knowledge – Web framework

Technological Pedagogical Content Knowledge – Web (TPCK-W) was developed as a result of the recognition that the Web opened new opportunities for the creation, distribution, and access to learning resources in quantities and varieties not previously imagined (Chou & Tsai, 2002), while also changing the nature of learning to facilitate both synchronous (traditional) as well as asynchronous and collaborative opportunities for teaching and learning (Neo, 2003). Focusing specifically on the Web as the Technology Knowledge integrated within the framework allows us to focus on the specific Web Knowledge (WK) required, including general knowledge about the use of the Web, Web-related tools, and Web-facilitated communications and interactions.

TPCK-W was developed and validated through data analysis of a study performed by Lee and Tsai (2010) that explored teachers’ self-efficacy in the Content, Pedagogical, and Web Knowledge bases, as well as their attitudes regarding Web-based instruction. It found that teachers who integrate more Web-based instruction have strong self-efficacy regarding TPCK-W and have a greater degree of positive attitudes toward Web-based instruction.

The Smithsonian integrated these frameworks into a reusable survey instrument in order to investigate how an awareness of these knowledge bases among users of Smithsonian educational resources could impact the development of platforms on which they are presented. An emerging project was used to develop and test this instrument.

6. Case study: Smithsonian Learning Lab

To examine the role of teacher knowledge of instructional design, TPCK-W, and the intersection of the museums’ role in educational impact, an analysis of targeted potential teacher-users of an emerging project at the Smithsonian (of which I am currently the project director) was conducted. This project, the Smithsonian Learning Lab, is a tool that places more than a million museum digital assets into an instructional design platform for teachers.

Since its founding in 1846, the Smithsonian has grown in physical size, adding new museums and centers, has grown to employ more than six thousand people, and has also seen substantial growth in nature of its audiences, both physical and digital. This sustained growth (primarily seen in our digital visitation) reflects not only the increasing work of those at the Institution, but the access desires of the people we serve. The Smithsonian will be, for the foreseeable future, more of a digital resource for information than a physical destination.

Smithsonian Learning Lab history

The Smithsonian Learning Lab project is the result of a substantial rethinking of how the digital resources from across the Smithsonian’s nineteen museums, nine major research centers, the National Zoo, and more can be used together for learning (Milligan, 2014). It aspires to make these resources more accessible and more useful to teachers, students, parents, and anyone seeking to learn more. The Learning Lab is a Web-accessible digital platform that enables the discovery, by teachers and learners of all types, of millions of digital assets (digitized collections, videos, podcasts, blog posts, interactives, etc.) from the Smithsonian’s galleries, museums, libraries, and archives. It contains research-based tools that aid its users in the customization of its contents for personalized learning. The project launched in public beta in late October of 2015 at


Figure 3: a teacher-created (Kate Harris) collection within the Smithsonian Learning Lab that incorporates Smithsonian and non-Smithsonian collections, as well as user-created annotations and student quizzes (

The most relevant feature of the Learning Lab for the purposes of this paper is its distribution of large-scale digitized museum collections within a learning resource design platform.

The Smithsonian and digitization

In aggregate, the Smithsonian collections contain more than 138 million objects and specimens, more than 2 million library volumes, and more than 153,000 cubic feet of archival material. Of the entire collection, only 9 percent of those museum collections, 33 percent of the library volumes, and 52 percent of the archival material will be digitized (Beauchamp, 2015). While those percentages may seem limiting when considered individually, the full digitization plan for the Smithsonian would result in 13,640,096 digitized objects and books and nearly 80,000 cubic feet of archival material. At the time of writing, there are more than 9 million digitized catalog records (of which, 1.4 million contain digital surrogates) available online.

To better understand our target users (teachers) in light of the educational frameworks, and how they might utilize such a substantial and growing set of resources within a system like the Learning Lab, the following methodology was developed.

Research methodology and instrument development


A central characteristic of the project design has been the involvement of teachers, beginning with the research phases through to the development/building of the project throughout the past year. While the functional requirements grew directly out of our understanding of teachers’ needs both from our own research and that of others working in the field, continual check-ins with teachers, as user testers, was a crucial component. An ongoing understanding of our users will be crucial to address shortcomings of the current project and direct future development. To do this, a reusable survey instrument was needed that could integrate the educational frameworks above and offer to the development team a holistic understanding of teachers as designers of technology-enhanced learning resources. The Smithsonian conducted a limited survey with a small focus group in order to test the potential validity of this new instrument (through the relation of its findings to larger groups studied in the literature). The survey was designed to understand teachers’:

  • Self-efficacy in the integration of technology into pedagogy (using the TPCK-Web Self Efficacy Framework)
  • Knowledge and self-efficacy in design practices (using the McKenney Ecological Framework and the Cviko, McKenney, and Voogt user roles)
  • Depth and frequency of the use of digital and digitized resources for instruction (using the TIDSR Survey on the Use of Digitised Resources)

Survey instrument

The project team developed an original twelve-question, three-part survey instrument to effectively measure these three areas.

Part one of the survey measures teachers’ self-efficacy in their own TPACK. To do so, we adapted the TPCK-W instrument developed and validated by Lee and Tsai (2010). This part of the survey groups questions into five sections: Web-general, Web-communicative, Web Content Knowledge, Web Pedagogical Content Knowledge, and Attitudes Towards Web-based Instruction.

Part two of the survey investigated both teachers’ self-efficacy and their roles as designers of technology-enhanced learning. We developed survey questions using the McKenney et al. Ecological Framework to examine self-efficacy (2015) and questions based on the three user roles defined by Cviko, McKenney, and Voogt (2014)—enactors, re-designers, and co-designers—to understand teachers’ typical roles as users and designers of technology-enhanced learning.

Finally, the third part of the survey explored frequency of teachers’ use of digital resources. The team adapted questions from the Oxford Internet Institute and UK Joint Information Systems Committee “TIDSR Survey on the Use of Digitised Resources (Meyer, Madsen, & Eccles, 2009).” The questions intended to explore the extent and frequency of participants’ use of a wide variety of digital tools and digitized online resources.

The focus group consisted of sixteen participants (classroom teachers) who completed the survey from July 14 to July 28, 2015. The complete survey instrument is available in Appendix I.


This results section is included primarily to offer a suggested approach for others who might use this survey instrument and analyze its results. A more detailed overview of the potential of this process in offered in the “Discussion” section below.

Teaching experience

As we will discuss later, the literature (Lee & Tsai, 2010) and the survey appear to indicate a clear relationship between teaching experience and attitudes towards the use of technology-enhanced learning, as well as skills in the integration of technology, pedagogy, and content knowledge. A majority (68.75 percent) of survey participants in this study would be considered highly experienced teachers, having more than fifteen years in the classroom, while more than 80 percent have more than ten years.

Self-efficacy in the integration of technology into pedagogy (via the TPCK-Web Self-Efficacy framework)

Survey participants reported high self-efficacy in the use of the Web for teaching in general areas (i.e., “click on a hyperlink”) as well as communicative ones, especially in consuming roles (“read others’ messages in a chatroom”). As well, survey participants’ confidence in their Web Pedagogy Knowledge was rated very highly. These questions focused on varying approaches in using the Web to discover and analyze appropriate content for instruction. The final and more complex triple integration of Web, pedagogy, and content knowledge bases, however, received the lowest efficacy ratings (although some individual questions that focused on the use of the Web to generally enhance teaching or increase student motivation did score highly). Table 1 summarizes the participants’ response in the five TPCK-W knowledge bases.

The final section of the survey measured the participants’ attitudes towards Web-based instruction. The responses here deviated very little as the participants reported near universally positive attitudes.

Knowledge base n (respondents) M
(standard deviation)
Web-general 16 5.92 0.27
Web-communicative 16 5.66 0.76
Web Content Knowledge 16 5.88 0.32
Web Pedagogical Content Knowledge 16 4.99 1.29
Attitudes Towards Web-based Instruction 16 5.76 0.50

Table 1: survey participants’ self-efficacy in the TPCK-W framework (rated on a six-point Likert scale)

Analyzing the survey responses in light of teaching experience shows slightly lower TPCK-W efficacy in participants with more than ten years of experience over those with less (5.50 and 5.98, respectively). Almost universally, those with fewer than ten years of experience rate their self-confidence as “extremely confident” for every question. Table 2 shows the full TPCK-W knowledge bases in relation to reported years of teaching.

0–5 0–5 6–10 6–10 11–15 11–15 15+ 15+ 0–10 0–10 11+ 11+
Knowledge base M S.D. M S.D. M S.D. M S.D. M S.D. M S.D.
Web-general 6.00 0.00 6.00 0.00 6.00 0.00 5.88 0.32 6.00 0.00 5.94 0.16
Web-communicative 6.00 0.00 6.00 0.00 5.25 0.75 5.64 0.69 6.00 0.00 5.45 0.72
Web Content Knowledge 6.00 0.00 6.00 0.00 5.90 0.10 5.84 0.36 6.00 0.00 5.87 0.23
Web Pedagogical Content Knowledge 6.00 0.00 5.94 0.06 4.63 0.63 4.80 1.42 5.97 0.03 4.72 1.03
Attitudes Towards Web-based Instruction 5.83 0.00 6.00 0.00 5.25 0.42 5.81 .042 5.92 0.00 5.53 0.42
Cumulative 5.97 0.00 5.99 0.01 5.41 0.38 5.59 0.64 5.98 0.01 5.50 0.51

Table 2: survey participants’ self-efficacy in each of the TPCK-W knowledge bases correlated to reported years of teaching experience

Knowledge and self-efficacy in design practices (via the McKenney Ecological Framework and the Cviko, McKenney, and Voogt user roles)

The first part of this section attempted to address the survey participants’ perspectives on the knowledge and experience required to be effective designers of technology-enhanced learning (self-awareness of the impact of their personal knowledge and experience). While, generally, the participants expressed high agreement with the statements, some variation can be seen (table 3). The lowest confidence, as well as the greatest variation in response, appears in the know-when section of the survey, whose questions focused on a teacher’s reactive, or non-reflective, practice: “In designing technology-enhanced learning, my decisions are more often based on the immediate needs of the context I am in rather than some form of formal reflective practice.”

Aspects of teacher knowledge Question focus n M S.D.
Know-what Articulation of personal design knowledge 15 4.53 1.26
Know-what Articulation of personal design knowledge limitations 15 4.50 0.95
Know-why Awareness of growth/change of personal design knowledge 15 5.27 0.93
Know-when Reactive decision making 15 4.47 1.41
Know-who Awareness of accessible design expertise 15 5.00 1.21
Know-where Awareness of successful personal design implementation 15 4.93 1.18
Know-where Awareness of unsuccessful personal design implementation 15 5.20 0.83

Table 3: survey participants’ perspectives on the knowledge and experience required to be effective designers of technology-enhanced learning, based on the McKenney Ecological Framework (McKenney et al., 2015).

As shown in table 4, the survey participants ranked their confidence as enactors and re-designers slightly more highly than in the role of designing resources both independently and collaboratively (although collaborative confidence was slightly higher than independent).

Role Question focus n M S.D.
Enactor Implementation of pre-made materials 16 5.25 1.35
Enactor On-the-fly adjustments to pre-made materials 16 4.81 1.55
Re-designer Adaptation of pre-made materials in advance 16 5.00 1.37
Co-designer Independent design of new materials 16 4.19 2.07
Co-designer Collaborative design of new materials 16 4.50 1.66

Table 4: survey participants’ reported self-efficacy in their roles as designers of TEL.

0–5 0–5 6–10 6–10 11–15 11–15 15+ 15+
Role M S.D. M S.D. M S.D. M S.D.
Enactor 6.00 0.00 6.00 0.00 4.50 1.50 5.18 1.40
Enactor 6.00 0.00 6.00 0.00 4.50 1.50 4.55 1.62
Re-designer 6.00 0.00 6.00 0.00 5.50 0.50 4.64 1.49
Co-designer 6.00 0.00 5.50 0.50 3.50 2.50 3.91 2.07
Co-designer 6.00 0.00 5.50 0.50 4.50 1.50 4.18 1.75

Table 5: survey participants’ self-efficacy in their roles as designers of technology-enhanced learning correlated to reported years of teaching experience.

As above in the TPCK-W results, we see an inverse relationship between years of teaching and independent or collaborative redevelopment or design of learning resources. Those with less than ten years report almost universal self-confidence in their abilities to use, re-design, and create learning resources, while those with more than 10 years report lower levels (table 5).

Depth and frequency of the use of digital and resources for instruction (via the TIDSR Survey on the Use of Digitised Resources)

The survey questions in this section were divided into two sets. The first examined the participants’ frequency of use of a wide variety of digital media—everything from e-mail to primary-source digital surrogates. A wide range of results was indicated, but the results at the high end of frequency tended to focus on general computer usage (word processing or e-mail), while those on the low end of frequency were in the areas of reference materials and media types that may only have narrow subject applicability (such as digital maps).

The second section looked specifically at the frequency of use of digitized resources (such as digitized primary sources, videos, etc.; i.e., the kinds of materials that the Learning Lab will make available). Here, high rates were revealed in the usage of this specific category of resource for the purposes of reference, personal research, and direct use in teaching. One participant explained, “The Web is a powerful resource that is easy to use. It also offers teachers the opportunity to find rich updated resources possibly not available in the past.” Lower rates were indicated mostly for quantitative analysis—again perhaps a narrowly applicable usage.

Activity Use category n M S.D
Teaching resource In teaching 16 5.19 1.13
Online reference Reference and research 16 5.00 1.37
Download for later use In teaching 16 4.75 1.39
Personal research Reference and research 16 4.69 1.10
Reuse, modify, combine In teaching (design) 16 4.44 1.90
Finding aid Reference and research 15 4.40 1.50
Resource for analysis Reference and research (vague) 16 4.00 1.66
Resource for quantitative analysis Reference and research (narrow) 16 2.88 1.41

Table 6: frequency of use of digitized resources by survey participants, sorted by highest frequency.


The intention of the case study survey was to attempt the development of a new survey instrument that could assist project teams, such as the Smithsonian Learning Lab team, to gain a deeper understanding of its teacher users. Specifically, how, using a much larger participant group, the project team could better understand how to examine the effectiveness of the Learning Lab after its launch. In general, it was assumed that those who score highly on the TPCK-W indicators and on the design framework are best positioned to make use of the Learning Lab, as it is a platform that combines the use of technology-enhanced learning with instructional design. Allowing us to filter for their confidence in integrating a technology like this in their pedagogy, their experience and knowledge in designing learning resources, and their existing use of digital and digitized resources gives us a model to replicate at a larger scale once the project is fully launched. An understanding of these three filters and how our potential users align to them will enable us to make more informed decisions on how to make the most appropriate adaptations or enhancements to align with the aptitudes or deficiencies of these users.

However, due to its limited scope, the study was not entirely useful in making specific decisions on how we should change the Learning Lab at this point; a substantially larger number of users of the Lab would be required to develop this understanding. The activity was primarily useful, in the contexts of this paper, to explore the three areas of investigation (self-efficacy in the integration of technology into pedagogy, knowledge and self-efficacy in design practices, and depth and frequency of the use of digital and digitized resources for instruction) and develop a replicable survey instrument for understanding future users. However, a few interesting results (in light of their relationship to the literature) did surface that warrant further discussion.

Impact of teaching experience

As we have seen in the literature, there is a clear correlation between self-efficacy in the integration of technology, pedagogy, and content knowledge and years of teaching experience (Lee & Tsai, 2010). In a limited study such as this one, we must be cautious to draw concrete conclusions from such a limited testing base, especially when looking at the results across years of teaching experience, as less than 20 percent of our survey participants had ten or fewer years of experience. The almost 80 percent of those who responded who had eleven or more years must be considered with the proper perspective.

If we look at the data separately, however (as we have done in the results section above), some clear patterns begin to emerge that do reflect the inverse relationship shown for TPCK-W success for teachers with higher levels of teaching experience. The results in table 2 match the Lee and Tsai findings as, across the five TPCK-W knowledge bases, participants with fewer years of teaching experience (zero to ten) universally rated their efficacy higher than those with greater experience (more than ten years). As well, when we look through the lens of the Cviko, McKenney, and Voogt user roles, we see lower self-efficacy in more experienced teachers in the roles that require more adaptation or creation of new resources (table 5), actions that we hope to encourage on the site. It may point to the need for additional professional development for this group to better understand how a tool like the Learning Lab can be integrated into their existing pedagogy (the literature and the survey both seem to indicate that they would not be opposed to professional growth in this area, as all indicate positive attitudes towards this integration).

Preparation to adapt and create

Are the potential users of the Learning Lab prepared to adapt existing learning resources and use the tools of the platform to create their own? Studies have indicated that the adaptation for the purposes of creating new learning resources, by teachers, is not a universal activity (Purcell et al., 2013). Only about 50 percent of teachers report that they engage in activities described as “take material found online—like songs, text, or images—and remix it into your own creation.” In this survey, the activity as described—“To reuse, modify and/or combine various resources to make something new or edit for new uses”—was engaged in “often” by fewer than one-third of survey respondents (mean response from 4.69/6). However, compared to the 12 percent of US adults who report engaging in these activities (in the Pew study), teachers are a natural user group for museums to target if they hope to encourage this usage of their collections.

Impacts for the Learning Lab

The three-part survey developed for this paper is unique in that it integrated an existing educational framework (TPCK-W) with an approach to investigate individual teacher aptitude in the role as designer. However, a larger testing group would be required to better evaluate its potential for developing a more unified understanding of how design knowledge and experience impact and/or integrate into the already complex unification of technology, pedagogy, and content knowledge, especially if it is to be used to weigh decisions in an audience-based methodology. The case study project, the Smithsonian Learning Lab, has been designed to be in “perpetual beta” (O’Reilly, 2005) so that its design and functionality can continually adapt to the needs and abilities of its users. To ensure that the project team can develop an appropriate understanding of the meaning of the observed usage (weighted according to things like their teaching experience and level of TPACK knowledge), it will be important to survey these users using this new instrument.

Future use of the survey instrument

This paper sought to develop a more complex instrument to understand teacher users of museum digital resources. Its structure builds upon the work of educational and design theorists to connect the academic or theoretical understanding of teaching practice with the practical: what we observe and know of our users. It may be, as it was designed and used in this paper, too complex for ad hoc surveying of users, so we might consider what could be learned by other institutions as they use this survey, or parts of it, with their own users. High self-efficacy in our users, in terms of the design of technology-enhanced learning and the integration of technology, pedagogy, and content knowledge, might enable us to take advantage of more complex technology tools and platforms in serving our users. It could suggest that institutional efforts might be better spent in improving the quantity, quality, variety, and accessibility of digitized museum collections than on developing applications of these resources (the users would be able to do this for themselves). Lower self-efficacy could point to the need for additional assistance and professional development in using these resources, as well as the development of best practices, from museum education departments, in how to best utilize digital museum resources within the classroom.

7. Conclusions

Museums hold in trust an endless supply of ideas, visions, and human mysteries to be unlocked for audiences of all kinds. It may be true that none of the museum’s several audiences is more frustrating or more difficult, but it is also clear that none is more important than teachers, none more worthy of all the energy, imagination, and intelligence the museum can command. (Newsom & Silver, 1978, p. 470)

Being audience driven is not just a fashion of the information age, but also the proper approach for museums to take if they are to remain relevant and useful into the future. However, as the world has continued to change, both for the worseincluding the rapid decline of funding available for school-based museum field trips (Ellerson & McCord, 2009)and for the bettersuch as the explosion of digital resources (including digitized collections) available to anyone anywheremuseums are in a position to readdress their core audiences to better understand their needs and the potential of our institutions to meet them. This unique opportunity centers on what is unique about museums: their objects and the unique characteristics of learning that is object-centered. This learning is personal, contextualized, and requires time (Rennie & Johnston, 2004; Falk & Dierking, 2000), all characteristics that technology-enhanced learning may be particularly capable of fulfilling.

What might it take for museums to completely become audience driven in a manner that addresses the huge potential we have in the education space, to move beyond improved discoverability and utility? Gurian (2013) addresses this possibility through an examination of the potential evolution of the school system into one in which the distinctions between formal (classroom-based) and informal (museum-based) lessen. Gurian’s theory of “organizational atomization” looks toward a future in which our institutions and the people they serve are capable of separating the physical assets (the schoolhouse, the museum) from the organizational mission. This differentiation would allow museums, for example, to offer simultaneous (both local: physically situated in the building; and global: virtually delivered through digital technologies) resources and services, independent yet reinforcing. In other words, the strict educational distinctions of the past could dissolve, and the museum could be a source for formal learning, one of the many “service providers” offering educational resources (along with the school, community members, etc.) to learners and the community. This “trans-border amalgam of services” (Gurian, 2013) for education, facilitated through digital technologies, is an exciting prospect requiring fundamental evolutions of many of our most traditional institutions.

The field of education (as an institution: its structure, teachers, etc.), however, may not be in the same post-digital status as Parry (2013) describes museums, where “digital normativity” (p. 25) has paved the path for bold rethinking. The post-digital teacher may have some catching up to do to be prepared to utilize the resources being made available by a “post-digital museum” (Parry, 2013, p. 24). Truly achieving what Oppenheimer, founding director of the Exploratorium, called “a parallel educational system” (as quoted in Newsom & Silver, 1978, p. 260), museums must continue to take steps in the right direction. Understanding users, especially teachers, from both theoretical and data-informed perspectives is crucial, as they are in a unique position to integrate museum digital resources into classroom instruction. Museums must be useful to them in order to achieve their educational missions.


The author thanks Stephanie Norby and Michelle Smith for their support and leadership in the development of the Smithsonian Learning Lab.

Thank you to the Digital Heritage, as well as the Distance Learning teams at the University of Leicester, for their support, advice, and encouragement (in particular Jeanette Atkinson, Ross Parry, and Giasemi Vavoula).

Thank you to the Smithsonian Learning Lab project team and leadership for their hard work, open sharing of ideas, and sustained enthusiasm for making Smithsonian digital collections discoverable and useful for teachers and their students:

  • For the Smithsonian: Claudine Brown, James Collins, Cody Coltharp, Michael Edson, Flint Hamilton, Pino Monaco, Ashley Naranjo, Stephanie Norby, Tess Porter, Rhett Ransom, Philippa Rappaport, Cara Seitchek, Michelle Smith, Tracie Spinale, Melissa Wadman, and Toni Willis
  • For Navigation North: Brian Ausland, Joe Hobson, and Dan Krieger
  • For Codename Design: Marc Baumgartner and Graham Dobson

Finally, a special thank you to Michelle Smith for her invaluable perspective and critical editing.

The title of this paper, Moving from Downloads to Uploads, references and pays homage to the work of Peter Lunenfeld in The Secret War Between Downloading and Uploading: Tales of the Computer as Culture Machine, which played a significant role in inspiring this research.


Abbott, F., & D. Cohen. (2015). “Using Large Digital Collections in Education: Meeting the Needs of Teachers and Students.” Digital Public Library of America.

Ash, D., & G. Wells. (2006). “Dialogic inquiry in classroom and museum: actions, tools, and talk.” Counterpoints 249: 35–54.

Baker, R. (2009). Museums & Education Digital Content Exchange Final Report, Curriculum Corporation.

Beauchamp, J. (2015). Collections and Digitization Reporting FY 2014, Highlights Summary – Smithsonian Institution. Washington, DC: Smithsonian Digitization Program Office/Smithsonian National Collections Program. 1.02.

Boschman, F., S. McKenney, & J. Voogt. (2014). “Understanding decision making in teachers’ curriculum design approaches.” Educational Technology Research and Development 62(4): 393–416.

Chan, S. (2009). “Will Schools Use Collection Content? The Learning Federation Pilot Report.” Fresh & New(er). Accessed August 4, 2015. Available

Chatterjee, H. (2007). “Staying essential – Articulating the value of object based learning.” Proceedings of the 7th Conference of the International Committee of ICOM for University Museums and Collections, 37–42.

Chou, C., & C. C. Tsai. (2002). “Developing web-based curricula: Issues and challenges.” Journal of Curriculum Studies 34: 623–636.

Clark, C., & R. Yinger. (1980). “The Hidden World of Teaching.” Annual Meeting of the American Educational Research Association. Boston, MA.

Cviko, A., S. McKenney, & J. Voogt. (2014). “Teacher roles in designing technology-rich learning activities for early literacy: A cross-case analysis.” Computers & Education 72(0): 68–79.

Davis, E.A., C. Beyer, C.T. Forbes, & S. Stevens. (2011). “Understanding pedagogical design capacity through teachers’ narratives.” Teaching and Teacher Education 27(4): 797–810.

Duhs, R. (n.d.). “An Introduction to Object Based Learning.” Object based learning. UCL Museums and Collections.

Eberbach, C., & K. Crowley. (2005). “From Living to Virtual: Learning from Museum Objects.” Curator: The Museum Journal 48(3): 317–338.

Ellerson, N.M., & R.S. McCord. (2009). One Year Later: How the Economic Downturn Continues to Impact School Districts. American Association of School Administrators.

Falk, J.H., & L.D. Dierking. (1992). The museum experience. Washington, DC: Whalesback Books.

Falk, J.H., & L.D. Dierking. (2000). Learning from Museums: Visitor Experiences and the Making of Meaning. Walnut Creak, CA: Altamira.

Falk, J.H., & L.D. Dierking. (2012). Museum Experience Revisited. Walnut Creek, CA: Left Coast Press.

Farnum, R.B. (1919). “The Place of the Art Museum in Secondary Education.” The Metropolitan Museum of Art Bulletin 14(9): 194–196.

Frost, C.O. (2002). “When the Object is Digital: Properties of Digital Surrogate Objects and Implications for Learning.” In S.G. Paris (ed.). Perspectives on Object-centered Learning in Museums. Mahwah, NJ: Lawrence Erlbaum Associates: 79–94.

Goode, G.B. (1891). The Museums of the Future. Washington, DC: Smithsonian Institution United States National Museum.

Gurian, E.H. (1999). “What Is the Object of This Exercise? A Meandering Exploration of the Many Meanings of Objects in Museums.” Daedalus 128(3): 163–183.

Gurian, E.H. (2013). “Expanding the known: a paper about the future of education and the museum’s role.” Omnium Gatherum: Occasional Papers. Accessed September 9, 2015. Available

Heyman, S. (2015). “A Museum at the Forefront of Digitization.” The New York Times. New York.

Howes, D.S. (2007). “Why the Internet Matters: A Museum Educator’s Perspective.” In H. Din and P. Hecht (eds.). The Digital Museum: A Think Guide. Washington DC: American Association of Museums: 67–78.

ISTE. (2008). ISTE Standards for Teachers, International Society for Technology in Education.

Jackson, P. (1966). The Way Teaching Is. Washington, DC: National Education Association.

Jackson, P. (1968). Life in Classrooms. New York, NY: Holt.

Kali, Y., S. McKenney, & O. Sagy. (2015). “Teachers as designers of technology enhanced learning.” Instructional Science 43(2): 173–179.

Kerr, S.T. (1981). “How teachers design their materials: implications for instructional design.” Instructional Science 10(4): 363–378.

Kirschner, P.A. (2015). “Do we need teachers as designers of technology enhanced learning?” Instructional Science 43(2): 309–322.

Laurillard, D. (2012). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York, NY: Routledge.

Lee, M.-H., & C.-C. Tsai. (2010). “Exploring teachers’ perceived self efficacy and technological pedagogical content knowledge with respect to educational use of the World Wide Web.” Instructional Science 38(1): 1–21.

Leinhardt, G., K. Crowley, & K. Knutson. (2002). Learning Conversations in Museums. Mahwah, NJ: Routledge.

Lenhart, A. (2015). Teens, Social Media & Technology Overview 2015: Smartphones facilitate shifts in communication landscape for teens. Pew Research Center.

Lindquist, T., & H. Long. (2011). “How can educational technology facilitate student engagement with online primary sources?” Library Hi Tech 29(2): 224–241.

Lohr, S. (2014). “Museums Morph Digitially: The Met and Other Museums Adapt to the Digital Age.” The New York Times. New York.

Lunenfeld, P. (2011). The Secret War Between Downloading and Uploading: Tales of the Computer as Culture Machine. Cambridge, MA: The MIT Press.

Matuk, C.F., M.C. Linn, & B.-S. Eylon. (2015). “Technology to support teachers using evidence from student work to customize technology-enhanced inquiry units.” Instructional Science 43(2): 229–257.

McKenney, S., Y. Kali, L. Markauskaite, & J. Voogt. (2015). “Teacher design knowledge for technology enhanced learning: an ecological framework for investigating assets and needs.” Instructional Science 43(2): 181–202.

Meyer, E.T., C. Madesn, & K. Eccele. (2009). TIDSR Survey on the Use of Digitised Resources. Surveys for Evidence of Impact. Oxford Internet Intitute, University of Oxford.

Milligan, D. (2014). “What Is the Smithsonian Learning Lab?” Smithsonian Learning Lab News. Accessed September 9, 2015. Available

Milligan, D., & M. Wadman. (2015). “From physical to digital: Recent research into the discovery, analysis, and use of museums resources by classroom educators and students.” MW2015: Museums and the Web 2015. Chicago, Illinois, MW2015: Museums and the Web 2015.

Mishra, P., & M.J. Koehler. (2006). “Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge.” Teachers College Record 108(6): 1017–1054.

Neo, M. (2003). “Developing a collaborative learning environment using a web-based design.” Journal of Computer Assisted Learning 19: 462–473.

Newsom, B.Y., & A.Z. Silver. (1978). “The Art Museum and the Young, Their Teachers, and Their Schools.” In B.Y. Newsom and A.Z. Silver (eds.). The Art Museum as Educator, A Collection of Studies as Guides to Practice and Policy. Berkeley, CA: University of California Press: 259–264.

O’Reilly, T. (2005). “What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software.” Accessed September 10, 2015. Available

Pamuk, S., M. Ergun, R. Cakir, H.B. Yilmaz, & C. Ayas. (2013). “Exploring relationships among TPACK components and development of the TPACK instrument.” Education and Information Technologies 20(2): 241–263.

Paris, S.G. (2002). Perspectives on Object-centered Learning in Museums. Mahwah, NJ: Lawrence Erlbaum Associates.

Parry, R. (2013). “The End of the Beginning: Normativity in the Postdigital Museum.” Advances in Research – Museum 1(1): 24–39.

Purcell, K., A. Heaps, J. Buchanan, & L. Friedrich. (2013). “How Teachers Are Using Technology at Home and in Their Classrooms.” Pew Research Center’s Internet & American Life Project.

Rennie, L.J., & D.J. Johnston. (2004). “The nature of learning and its implications for research on learning from museums.” Science Education 88(S1): S4–S16.

Rice, D., & P. Yenawine. (2002). “A Conversation on Object-Centered Learning in Art Museums.” Curator: The Museum Journal 45(4): 289–301.

Roberts, S. (2014). “By Digitizing Images, Museum Opens a Window Into the Past.” The New York Times. New York.

Sanderhoff, M. (2014). “This belongs to you: On openness and sharing at Statens Museum for Kunst.” In M. Sanderhoff (ed.). Sharing is Caring: Openess and sharing in the cultural heritage sector. Copenhagen: Statens Museum for Kunst.

Shulman, L.S. (1986). “Those Who Understand: Knowledge Growth in Teaching.” Educational Researcher 15(2): 4–14.

Siegal, N. (2013). “Masterworks for One and All.” The New York Times. New York.

Silver, A.Z. (1978). “Issues in Art Museum Education: A Brief history.” In B.Y. Newsom and A.Z. Silver (eds.). The Art Museum as Educator, A Collection of Studies as Guides to Practice and Policy. Berkeley, CA: University of California Press: 13–18.

Smithsonian Institution Archives. (2009). “The Children’s Room in the Smithsonian Institution Building: Knowledge Begins in Wonder.” Retrieved July 28, 2015. Available

Sotto, T. (2012). “Summary Report of Survey Findings, Teacher Programs: Assessing the Getty Museum’s Online Resources for K–12 Teachers.” J. Paul Getty Trust.

Taylor, P. (1970). How Teachers Plan Their Courses. London: National Foundation for Educational Research.

Taylor, W. (1980). “Teacher and materials: the selection process.” In M. Cambre (ed.). Secondary School Video. Bloomington, IN, Agency for Instructional Television.

Voogt, J., P. Fisser, N. Pareja Roblin, J. Tondeur, & J. van Braak. (2013). “Technological pedagogical content knowledge – a review of the literature.” Journal of Computer Assisted Learning 29(2): 109–121.

Weil, S.E. (2007). “From Being about Something to Being for Somebody.” In R. Sandell and R.R. Janes (eds.). Museum Management and Marketing. New York, NY: Routledge.

Williams, D. (2010). “A Brief History of Museum Computerization.” In R. Perry (ed.). Museums in a Digital Age. London and New York: Routledge: 15–21.

Wolfrom, K.A. (2010). “The Rise and Fall of the Philadelphia Commercial Museum: How a Forgotten Museum Forever Altered American Industry.” Independence Seaport Museum. Accessed July 30, 2015. Available

Yinger, R. (1980). “A study of teacher planning.” Elementary School Journal 80: 108–127.

Appendix I: Survey Instrument


Number of years teaching?

  1. 0–5
  2. 6–10
  3. 11–15
  4. 15+

Section 1

Please rate your confidence in performing the following actions.
A rating of 1 means you feel not at all confident in performing the action, and 6 means you feel extremely confident.


  1. I am able to click a webpage hyperlink to connect to another website.
  2. I am able to key in a website address to connect to a particular website.
  3. I am able to print out the contents of a website.
  4. I am able to search for information on a website using keywords.
  5. I am able to download pictures from the web.
  6. I am able to use search engines.
  7. I am able to copy text from a webpage into another application (like Word).

Please rate your confidence in performing the following actions.
A rating of 1 means you feel not at all confident in performing the action, and 6 means you feel extremely confident.


  • I am able to read others’ messages in a live chatroom/discussion forum.
  • I am able to set a username by yourself in a live online chatroom/discussion forum.
  • I am able to talk to others one on one in a live online chatroom/discussion forum.
  • I am able to provide information or respond to someone else on a web-based bulletin board system (non live).

Please rate your confidence in performing the following actions.
A rating of 1 means you feel not at all confident in performing the action, and 6 means you feel extremely confident.

Web-Content Knowledge

  • I know that the web can provide various materials to enrich course content.
  • I know how to search online resources for course content.
  • I know how to select appropriate content from web resources.
  • I am able to search related online materials to locate complete course content.
  • I am able to search for various materials on the web to be integrated into your existing course content.

Please rate your confidence in performing the following actions.
A rating of 1 means you feel not at all confident in performing the action, and 6 means you feel extremely confident.

Web Pedagogical Content Knowledge

  • I know how to apply teaching modules on the web into courses.
  • I am able to use web technology to enhance teaching.
  • I am able to use the web to enhance students’ learning motivation.
  • I am able to select proper existing web-based courses to assist teaching.
  • I am able to apply web technology to support multiple teaching strategies on a particular course unit.
  • I am able to guide students to use web resources to study a certain course unit.
  • I am able to use web resources to guide students’ learning activities for a certain course unit.
  • I am able to use web technology to support teaching for the content of a particular course unit.

Attitude toward web-based instruction

Please rate your attitudes on the following statements.
A rating of 1 means you strongly disagree with the statement, and 6 means you strongly agree.

  • Web technology can be used in the practice of teaching.
  • The unique characteristics of the web can help instruction.
  • Web technology can enhance teaching skills.
  • Web-related resources can enrich course content.
  • Web-based teaching can enhance students’ learning motivation.
  • Web-based teaching is a future trend in education.

Please answer the following question.

What are your opinions, both positive and negative (if applicable), about integrating the web into your teaching?

Section 2

Technology-Enhanced Learning

Please rate your perspective on the following statements related to the integration of technology into instructional design.
A rating of 1 means you strongly disagree with the statement, and 6 means you strongly agree.

The term “technology-enhanced learning” combines all approaches in which technology is used to support the learning or teaching process (including e-learning, online learning, game-based learning, web-based learning, and others).

  • I could articulate the specific knowledge and experience I use when designing technology-enhanced learning.
  • I could articulate the specific knowledge and experience I lack when designing technology-enhanced learning.
  • I am aware of how my practice in developing technology-enhanced learning has changed with experience.
  • In designing technology-enhanced learning, my decisions are more often based on the immediate needs of the context I am in, rather than some form of formal reflective practice.
  • I know which colleagues or what resources to consult to assist me while I am designing technology-enhanced learning.
  • I am aware of where, in the curriculum I teach, my efforts to create technology-enhanced learning thrive.
  • I am aware of where I struggle in my efforts to create technology-enhanced learning.

Technology-Enhanced Learning II

Please rate your confidence in performing the following actions.
A rating of 1 means you feel not at all confident in performing the action, and 6 means you feel extremely confident.

  • I can implement ready-to-use technology-rich activities.
  • I can make subtle adjustments to ready-to-use technology-rich activities, as needed, on the fly.
  • I can adapt ready-to-use technology-rich activities and materials for your current curriculum.
  • I can independently design completely new technology-rich learning activities and materials for your class.
  • I can collaboratively design completely new technology-rich learning activities and materials for your class.

Section 3

Use of Digital Resources

Please rate how frequently you engage in the following activities.
A rating of 1 means you never engage in the activity, and 6 means you often engage in it.

  • Word processing tools (MS Word, Google Docs., etc.)
  • E-mail
  • Internet search engines (Google, Yahoo!, etc.) but NOT Google Scholar
  • Google Scholar
  • CD-Roms
  • Online library catalogues
  • Online reference sources (dictionaries, Wikipedia, etc.)
  • Online databases
  • Digital facsimiles of historical materials (primary sources from museums, libraries, or archives, etc.)
  • Data archives
  • Digital audio materials (speeches, interviews, oral histories, music)
  • Digital film or video (including from YouTube)
  • Digital maps
  • Blogs

Use of Digitized Resources

Please rate how frequently you engage in the following activities related to access and use of digitized resources.
A rating of 1 means you never engage in the activity, and 6 means you often engage in it.

Digitized resources refer to a range of materials online—from scanned texts and images of objects, resources that historically would been accessed in person: in a library, archive, or museum, but have become digitized and made available for broader access.

  • As an online reference source
  • To download to my computer to use offline
  • To find materials I later consult in person
  • As a teaching resource
  • As a research resource that I analyze manually
  • As a research resource that I analyze quantitatively (such as computational analysis of text)
  • To pursue personal interests or research
  • To reuse, modify and/or combine various resources to make something new, or edit for new uses

Cite as:
Milligan, Darren. "Moving from downloads to uploads: Toward an understanding of the curricular implications of access to large scale digitized museum collections on the professional practice of K–12 classroom educators." MW2016: Museums and the Web 2016. Published January 15, 2016. Consulted .