ecce – Since 1 June, ecce has been working with the Dortmunder U on evaluating the festival "Sommer am U" [Summer at the U]. How did this cooperation come about?
Jasmin Vogel (JV): A tagline for the festival "Sommer am U" is "Made by Many". We offer a platform for the local scene. For three years, we have been posing the question: Are we achieving the aims? The festival was to provide something that has been absent so far in Dortmund and which would make a contribution to the Union quarter and to the cultural life here in Dortmund. All we have though is a gut feeling, but nothing substantial that we can really rely on. Then Nadine came to me and asked whether we should carry out a detailed survey on the dynamics between visitors and protagonists to get an idea of the extent as to which we are achieving our aims.
Nadine Hanemann (NH): We have been doing spillover research since 2014, and last year we had various case studies tested as part of a European framework. Then I thought we could use two or three methods being applied there very effectively to a festival. Jasmin had attended several workshops and was therefore familiar with the research process, so we began talking. The methods make sense and we could apply them – on a small scale of course – and then try them out in the Ruhr region.
ecce – ecce is located at the Dortmunder U. Are there or have there been other cooperations?
JV: There have been several cooperations. At the Forum d'Avignon Ruhr, we had the opportunity of presenting our own projects and we also cooperated closely, for example, on the Innovative Citizen project. There were several research groups at ecce in which we were involved. Events planned by ecce have frequently taken place at our premises. Therefore, we are working together very closely, and have been doing so since 2010.
ecce – Let's go back to the cooperation for the festival "Sommer am U". What precisely has been evaluated?
NH: For one thing, important information about the visitors of the festival. This refers to age, where they come from, what interests they have, what media has been used, how the visitors are culturally active in general. This gives us an idea about who we are addressing. For another thing, we wanted to find out the effect that this event has on the Dortmunder U, e.g. whether the festival results in more people visiting the location. And also the effects that the festival has on the city. Of course we also ask for suggestions on improvements and expectations. Certainly, there is a desire to move forwards.
JV: A primary question is the extent as to which the event is relevant to Dortmund. One should consider that 50% of the festival is funded by sponsors. Such a festival is not essential for cultural institutions such as the Dortmunder U: There are sufficient private initiatives. Therefore, the question to ask is one of the extent as to which Dortmund lacks such an event. This question frequently needs to be posed and tested considering the limitations on budgets. Are there already formats doing this that are not funded by the city?
A publicly funded institution should not compete with the private economy.
NH: We are also surveying the people involved, i.e. the programme makers, participants from previous years. What are their aims? What type of target group are they aiming for? This enables the examination of what is desired and what is realistic. Is there a match between both? What can be done additionally and how can the protagonists be assisted in moving forwards. What are their expectations for the future?
JV: Such festivals are actually based on individual effort. We currently have 26 people who are doing an amazing amount of work – and they are definitely not well rewarded for it. A recurring question that needs to be addressed is how to create a framework in which people want to remain active. And this is to be seen as a cooperation and not as the provision of a service. In this respect we have to critically ask ourselves whether we are using the cultural scene too much as a service for us. This can only be answered by honest feedback from the participants.
ecce – It’s therefore more about quality than saying we have attracted this many visitors?
NH: Most definitely. It's about gaining knowledge...
JV: ...not about statistics.
NH: Exactly. And we have to add that we are not carrying out a representative survey. We aim to reach around 20% of the expected visitors. Nevertheless, these are only trends.
ecce – How are the results evaluated and reported? Is there a final report available for the public?
NH: This hasn't been decided yet.
JV: It's still all beta! (laughs)
NH: We are of course evaluating the visitor survey. I will submit the results which we will discuss afterwards. It is of course based on statistical programmes, but there are also analyses of content and categorisations from which typical word clouds will emerge. These will form the basis for discussion.
JH: In any event, we present the results to the event organisers as they have participated in the survey. But I have to add, it is only a trial run for us. At the Dortmunder U, we have only been carrying out evaluation in the form of an Ipad survey so far. This provides us with some data but ultimately we would like to have a better mode of evaluation. It's not about pushing bureaucracy to its limits but rather testing whether what we're doing is right and relevant. Such a large institution is unable to achieve this from one day to the next. What's needed are smaller tests such as the festival "Sommer am U".
NH: We can't say yet whether we will be publishing a report. But this certainly makes sense for other cultural institutions in the region. It will enable us to learn from each other.
ecce – ecce operates at an interface between the Ruhr region and Europe. Which role does Europe play in this project and in the research area?
NH: This is important especially in the fields of research and and evaluation as it is the only way to receive input from the outside. England and the Netherlands are important actors who work with and develop innovative, new research models. We get lots of inspiration from this cooperation: new studies, current case studies... On the other hand, you do not just want to do research. Similar to the exchange between regional institutions, it is also extremely helpful to learn from European institutions about questions such as: Where is the potential? What can be transferred to other countries? Over the past 2 to 3 years, we have been active in this European research cloud which has delivered an incredible amount. It is now necessary to go out into the regions and ask what benefits can be drawn from this network and how it can be applied to the Ruhr region.
ecce – This means that the many years of cooperation are bearing fruit, specifically for the Dortmunder U?
NH: Absolutely! Last year, we had four case studies researched in Europe. That's important and interesting, but even more exciting is the question of their applicability to other cultural organisations and projects.
JV: Of chief interest is the application of what has been learned. Ultimately, it should be transferable to cultural institutions. This is best done when such institutions are local. In this way, we learn directly from each other, also on matters of staffing levels. Such an evaluation needs to be carried out by somebody. Someone needs to hand out questionnaires prior to each event, and someone has to collect them afterwards. There are many procedures that are easier to learn when things are local.
ecce – Organisation is one aspect. How do you achieve this within your own structures? Another aspect concerns mind sets. Is there a risk of becoming fixed on evaluation with possible consequences on the type of programmes being offered?
JV: There are different types of evaluation. For German cultural institutions, it's always about numbers. It's therefore not about specific events and generally there is no differentiation being made between important and less important factors. Theoretically, a workshop on culture can be more important than, for example, a major exhibition on Impressionism: The exhibition might well attract a hundred thousand visitors, most of them tourists, thereby having an impact on the economy but only with little sustainable impact on the city itself. Such aspects have been debated for a long time but are rarely transferred into concrete evaluation models.
ecce – Does using well-designed evaluation strategies provide an opportunity to go beyond the focus on visitor number to sustainable, long-term action?
JV: Yes. It is important to consider your own approach and aims. I think this is important. Every city thinks differently, also in the Ruhr region – Dortmund is different from Essen. For the festival "Sommer am U", we sat down and asked ourselves: What is relevant for Dortmund? Especially concerning this festival.
NH: I absolutely agree. Visitor numbers can be used as an indicator. But the problem with indicators is that you need to be careful to not just create controlling mechanisms. Indicators should help with decision-making in respect of project organisation. But how does it help when I say I must now fulfill these five indicators? This doesn't really make sense. It's more about reflecting on what you are doing. It can become counter-productive if you only work with visitor numbers without questioning the statistics. How do 100,000 visitors support my goals when these are not the visitors I want to reach?
JV: Plus, evaluations are not only important at the end of a process but also during the process. And that takes us back to the strategy. What is often missing is a long-term strategy for evaluation in which I select the format appropriate to the topics and aims that I want to achieve.
ecce – That's an interesting approach because many consider evaluations just to be about visitor and occupancy numbers. The pressure to achieve quantity-based success has the negative effect of cultural homogenisation, such as the blockbuster format.
JV: That's a strategic problem. If I allow myself to go in this direction, my strategy is wrong. And that's why I appeal to the museums to become active and not to use such statistics as benchmarks to compare themselves to other institutions. This is a task of the German Museums Association [Deutscher Museumsbund]. If we look at jobs in visitor research, other than at the Prussian Cultural Heritage Foundation [Stiftung Preußische Kultursammlung] there's nothing. Few people ask themselves the question whether what we're doing is relevant. Evaluation means being prepared to put yourself under examination.