The Digital Culture and Communication section of ECREA
Co-authored by Dr Annette Naudin & Dr Natalie Hart, Birmingham Centre for Media and Cultural Research, Birmingham City University.
In this paper, we interrogate a methodology for collecting and presenting data deployed to address inequalities in children and young people’s cultural engagement. Drawing on a case study, we explore the challenges of using data visualisation as a tool for supporting cultural organisations who seek to address inequalities in cultural provision. We are concerned with the robustness of the data, the ‘story’ the data visualisation presents, how this is used and interpreted by cultural organisations. The context for this study is an increasing pressure to demonstrate the impact of arts on people’s lives and to make use of data to inform cultural policy. As researchers working closely with the commissioning arts organisation, we reflect on the management processes and tensions which emerge from introducing new or different ways of working. Our research findings suggest two key outcomes. Firstly, that many arts organisations who are making use of data to propose new practices, are not adequately resourced to adopt and sustain tools for collecting, managing and sharing large sets of data. Secondly, an over celebration of the power of data visualisation tools, obscures the limitations of the ‘story’ they present and of the reliability of the data itself.
Data and the arts sector
The use of data as evidence of cultural engagement is increasingly important for cultural organisations who seek to justify public funding (Andrejevic et al, 2015). As Matthew Brown (2017) notes in his blog for Arts Professional “Measuring social value has become an increasingly prominent issue as funders expect recipient organisations to demonstrate how their work makes a difference. In some cases, funders are embedding social value measurements into their commissioning processes” . According to the Arts Council England, data on children and young people supports evidence based policy making and is a key tool for cultural organisations seeking to make a high-quality cultural offer. However, when the arts seek to harness the power of big data, they encounter the issue of measuring deeper impact, as Brown (2017) notes, “[…] measures of success need to transcend simple volume metrics. Measures such as audience numbers and number of participants are output measures. They describe the reach of the organisation but not necessarily its impact.” Lilley and Moore see this as partly due to an attitude within the sector, “Too often, the gathering and reporting of data is seen as a burden and a requirement of funding or governance rather than as an asset to be used to the benefit of the artistic or cultural institution and its work.”
The case study is a project which focuses on an organisation which seeks to demonstrate children and young people’s cultural engagement, by using data to map regional activities. The commissioning organisation, Arts Connect, is one of several Arts Council England funded ‘bridge’ organisation which create links and support relationships between the cultural sector and schools in their region. Our methodology has involved working on the project, interviews with Arts Connect staff, undertaking user-testing interviews and questionnaire for cultural organisations. As the project has not been completed at the time of writing, this paper reports on emerging findings and it is an opportunity to reflect and analyse the work to date.
In our user-testing interviews we found conflicting needs from Arts Connect’s partners, schools and cultural organisations, ranging from the need for a simple practical tool which can make the user’s life easier to the need for an aesthetically pleasing tool.
‘If you’re looking for, boom, boom, boom, why not visit or why not click here and then it goes straight to it […] bigger, in your face as well.’
For some, the tool was seen as a threat to their own knowledge and experience, challenging their current practices and adding an unwanted layer to their work.
…’I’ve been here for 36 years, I’m fairly familiar with what you’re showing me, with respect.’
In the context of current cuts to the arts & limited funds, organisations are concerned about the purpose of collecting yet more data and proposing new ways of working. We came across questions such as ‘Why are they making us do this? Will we be judged in a competitive funding environment?’ indicative of a general malaise.
‘[…]they’re just getting on and doing their own thing…Everybody’s trying to survive, so that’s always going to be the case.’
A reliance on networking and individual expertise, characterises the sector and there is a sense that instead of using a tool driven by data, individuals preferred to contact people they know and target schools based on previous knowledge. There were concerns about people engaging with the tool as part of their work and contributing to the process of collecting. This led to questions about the reliability and robustness of the data visualisation.
‘I would be cautious about the expectations around take up and usage …’
The reality for the bridge organisation is that data is hard work. Aside from creating a survey which arts organisations will be willing to fill in and which does not seem to be too time consuming, our findings suggest there are deeper problems. The bridge organisation visualised an open data approach which would involve the various partners, schools and freelancers contributing their data, regularly and accurately through an online portal. However, advice from the data visualisation designer is that this was too messy and unrealistic. The alternative is for the bridge organisation to get much more involved in collecting, vetting and regularly updating the data. The concern for the bridge organisation is the ability to manage this type of project, having assumed it would be very low maintenance.
So far, this study points to the challenges of introducing the notion of arts organisations using Big Data as an aspect of their work. The use of Big Data to demonstrate inequalities and support cultural organisations is seductive but in undertaking that work, there are several barriers. Workers in the sector feel inadequate, people are defensive about what they do, there are anxieties about being compared with others on a public visual tool and evidence of a level of fatigue with collaborating and responding to the need to collect data and share it. There are conflicting agendas which in this project are illustrated by the bridge organisation commitment to addressing inequalities in young people’s cultural engagement across the region and individual cultural providers, merely seeking to meet their own priorities. There are many instances when cultural organisations collaborate but they are increasingly operating in a competitive environment in which cuts to arts funding are adding new challenges.
There is an assumption that data, particularly open data, has the potential to provide a short cut to addressing inequalities in provision, by simply mapping and visualising current practice. However, data is only useful if it is relatively timely, reliable, and accessible. In a context in which data has often been used for advocacy, there are mixed feelings about the degree to which any data collected by arts organisations is robust enough. Yet, the visualisation of data could provide a space to begin a debate, think about what it tells us and what might be missing.