Measuring Impact of Non-Traditional Research Outputs

While traditional outputs include scholarly books, chapters, articles and conference papers, there are also an increasing variety of non-traditional research outputs. A research output is any form of materials embodying research, whether produced by writing, making, composing, designing, performing, or curating. (adapted from University of Sydney Guidelines for non-traditional research outputs)

Measuring the Impact of Creative and Performing Arts

“The Creative and Performing Arts Panel welcomes research outputs that embody original research produced by practitioners who are independently or collaboratively engaged in the creation of artistic works across the breadth of creative and performing arts disciplines in the subject areas of design; music; literary and other arts; theatre; dance; film, television and multimedia; visual arts and crafts”  (PBRF Panel Specific Guidelines, 2018).

Creative and performing arts research outputs may include works such as musical compositions, dance performances, film or video productions, creative written works, photographic exhibitions, computer software, or screenplays.

As with all types of academic research, creative practice is original, speculative and systematically pursues new knowledge, and may involve engaging in critical reflection, examination, and analysis of both one’s own processes of creative production and the wider social, cultural, theoretical and conceptual contexts in which this occurs.

The ephemeral nature of some creative works such as performances or exhibitions can present a challenge for impact measures.

The range of impacts listed below illustrates the wide variety of areas in which impacts from research across the Creative and Performing Arts may be found to have a clear influence on the quality of life of individuals and communities locally, nationally and internationally:

• the enrichment of cultural life and public discourse through the creative and performing arts
• increased community access and enrichment of cultural experiences through pre-event talks, programme notes and other public and media commentary associated with performances, exhibitions, screenings or broadcasts
• contributions to processes of commemoration, memorialisation, reconciliation and cultural development
• the impact of site-based art practices on how audiences engage with issues such as environmental politics, the histories of contested sites or the politics of land and place
• the impact of hui ā iwi focusing on whaikōrero, mau rākau or tā moko as customary art forms that shape, sustain and advance Māori knowledge and development
• the impact of art, design, music, literature and the performing arts on the profile of New Zealand culture and society internationally
• contributions to innovation and entrepreneurial activity through the design and delivery of new products and services
• contributions to economic prosperity via the creative sector, including publishing, music, theatre, museums and galleries, film and television, fashion, tourism and computer games
• provision of expert advice to governments, non-governmental organisations, charities and the private sector that influences policy and/or practice
• economic benefits generated by design using new technologies
• increased public involvement in literary, musical and other forms of creative endeavour
• the impact of new expert systems that facilitate innovative community participation, organisational change, financial services and organisational communication
• the contribution of artistic practices to public understanding of human rights and standards of health and wellbeing.

Software and Code

It is increasingly common for researchers to be producing or writing software or code as part of their research. Examples include:

• Apps
• New programming languages
• New operating systems
• Algorithms

Software files can be published and cited in the same way as any other research output, however, poor standards and norms for doing this and indexing the resulting citations, are challenges for tracking impact in this way. Publishers and database providers are improving the way full-text files are searched in order to uncover previously invisible mentions, for example in methodology sections and acknowledgments.

In addition, code repositories such as GitHub not only host these types of output but also display how many people have forked a given piece of software - that is, used it as the basis for a new project. Social media mentions, "stars" and other engagement can also be tracked.

Other examples of impact for software and code include uptake and use cases, commercialization, patents and citations in technical reports.