News and Discussion

Launched in 2010, The Ticketing Institute’s aim is to give commentary and a place for discussion on the new opportunites for ticketing, marketing and CRM, for digital marketing and social media integration.

Our news and discussion area is a home to press releases on stories affecting the industry as well as our own blog posts on the issues, trends and news of Sports and Entertainment ticketing / customer engagement.

Chiefs Select Secutix

 

EXETER CHIEFS APPOINT SECUTIX AS ITS NEW TICKETING PARTNER 

 

 

Club Has Partnered With The Cloud-Based Ticketing Platform To Support Their Digital Ambitions

 

London, Lausanne, Paris, Madrid – 12 October 2017: English rugby club and current Aviva Premiership Champions Exeter Chiefs  have appointed SecuTix as its official ticketing technology partner.

Following a competitive tender process, SecuTix, the global provider of a ticketing engagement platform for the sports, culture and entertainment industries, has begun work across the Chiefs’ two businesses: the rugby club itself and their stadium, Sandy Park.  Alongside being the home ground for the rugby club, Sandy Park is a successful conference and banqueting centre.

The Club will use the cloud-based SecuTix 360° software to:

  • Grow their online ticketing business
  • Increase the average basket value through both upselling and cross-selling
  • Offer an enhanced fan experience when purchasing tickets online with 3D seat mapping technology, thanks to SecuTix’s ability to integrate with PACIFA technology
  • Develop deeper relationships with their corporate audience through an improved hospitality customer journey
  • Gain a 360-degree understanding of their fans and be able to engage with them across digital and mobile touchpoints

 

Tony Rowe OBE, Chairman & CEO of Exeter Chiefs, said: “We wanted to partner with a modern technology ticketing company that could support our digital ambitions for both the Club and Sandy Park. The SecuTix team impressed us with their experience of ticketing and the rugby sector.  Together we will use data to better understand what the fans want, create a first-class experience for them and drive a commercial return for the Club.”

Frédéric Longatte, CEO of SecuTix commented: “We’re very much looking forward to working with such a top-flight team as the Chiefs.  We’re confident that our SecuTix 360° platform, which is delivered as ‘software as a service’ (SaaS), will benefit the Club across not just ticketing, but also CRM and digital marketing.  It’s exciting times with plans in place to extend Sandy Park to 20, 000 seats and SecuTix will support the Club throughout this period of expansion to help them realise their vision for the future.”

Exeter Chiefs sell over 200,000 tickets annually.  It is the second Premiership Club to partner with SecuTix, Saracens being the first back in 2016.

 

About SecuTix

SecuTix is a European technology provider of a Ticketing Engagement Platform that helps organisations boost ticket sales and enhance audiences’ experience before, during and after live events. Our product, SecuTix 360°, is a cloud-based platform that combines ticketing and marketing functionality, and is offered as a white label SaaS service. Used by the largest sport clubs and stadiums, live entertainment businesses, and leading museums and cities across Europe, SecuTix manages the yearly sales of 30 million tickets. Among our clients are Opéra National de Paris, UEFA, Centre Pompidou, Aspro Parks, Saracens RFC, Paléo Festival, Musée Picasso Paris and more. A daughter company of the ELCA Group, SecuTix has a local presence in Switzerland, France, Spain and the UK.

 

 

Why I Changed Ticketing Systems – A Consultant’s View

Many of you reading this will be responsible for hundreds of thousands, possibly millions of tickets per annum, perhaps realising ten of millions of pounds in revenue. You sell theatre shows, sports events and much much more.

 I wanted to share a story of changing ticketing systems on a MUCH smaller scale.

 We recently migrated ticketing platform, I say platform, should be more like plug in, for the Ticketing Professionals Conference registration.

“your website is broken” – sound familiar?

 Sounds easy doesn’t it? 400 places, two price codes, a few promotions and one general admission event.

 Embarrassing

It was embarrassing, that as a conference for Ticketing Professionals we had a system so prone to, how shall I put it, moments of……er……. oddness. I remember receiving an email from Liz Baird of Wales Millennium Centre in early 2016 struggling to book her place. When I looked into the issue, she had attempted on less than 5 or 6 times to register. No payment, just register.

 It’s embarrassing we can’t get ticketing right isn’t it?”

We sorted her registration and I emailed an apology and said “It’s embarrassing we can’t get ticketing right isn’t it?”

“Yes, a bit” she replied.

 We carried on for 2016 and ran the system through the 2017 conference booking process, but it became clear as we entered our busiest period, things would have to change, both for our own sanity, but also our image.

 

Support for $139 / Annum?

Away from the public facing issues we experienced, we also had some real back end issues. Really basic things, like wanting to clearly and efficiently show VAT (UK sales tax) in our pricing structure and to calculate and present it properly during the transaction.

 We were using a simple annual licence of a WordPress plug in, we are still using a plug in, just a different one, as part of that there are support strands, normally by forum. I don’t think, for the price you can expect much more now can you?

 I got constantly frustrated by what seem as apparent answers being vague or pushing the issue to another forum, product or issue. Something as basic and making sure a customer got a confirmation email became a long and drawn out affair.

 Whatever platform you use, for whatever genre, there are some things that it should just DO. Printing a ticket? Sales Summary Report? Client Record? ( the list goes on here )

 

Business being held back

So we had the public facing issues, the back end issues, which, to a certain extent we could manage, but we began to feel we were missing out on functions or rather business benefit from functions.

 What do you mean by that? Well here’s a perfect example – our old system supported promotion codes (for a extra fee) but they were for all price codes in one event.

 So when we want to do a special promotion for venue, but not for vendor registrations we could not do that. Again it sounds small, but it was holding us back, we either had to apply the promotion code to everything and try and plaster advertising that it was only valid on a certain ticket and check every booking or not run the promo. Either way it was holding us back.

 

Greener Grass?

Unlike a lot of you, there was no complex data migration or mass staff training, but there were still some key tasks to undertake.

One of the key ones was to integrate with our (yet constructed) website. We eventually chose a theme that was designed for conferences and optimised to work with our new platform. I guess many of you may have done the same – ‘what other orgs have you worked with / have you experience of integrating with XYZ Ticket system?’

The great advantage of this step to bring things together is that we can easily drop widgets throughout the site, something we often see in AudienceView sites, and one of my stand out favourite advantages of that system.

Easy to Add Widgets – A Great Feature

We have lost features though, some we did not realise that were really useful to us. We used to have a great API to Mailchimp to add new delegates to our mailing list and flag their attendance. This allowed for segmented mailing to target or exclude booked attendees, we have had to resort to a manual process.

 This highlights that any change will see advantages, but likely loss of a feature or business benefit

 

Worth It?

Heh, we still have some back end lifting to do with our new platform. The number one issue we wanted to resolve was a tighter web integration, better online customer experience and fewer “your website is broken calls” – sound familiar?

 As a venue operator I changed system four times, all with net positives, I did 60+ transformations as a vendor and around 25 as a consultant, so I can say I have experience of it!

 It was a great experience to go back to the coal face and experience the frustrations or users, agony of research and the pain of implementation. On the back end of the project, I can’t imagine using anything else, I am thoroughly happy. Now, back to the business of using it to sell tickets………. While your here, have you booked for #TPC2018 yet?

 Check out our engine and get your place here (shameless plug over)

 

 

Yesplan on Tour Event 21st November, London

Yesplan on Tour Event – London 21st November

Cloud based venue management system Yesplan are bringing their ‘Yesplan on Tour’ series to the UK on the 21st November.

The event will be at Flanders House, the Belgium Embassy in Cavendish Square, London and run from 1.30 until 430pm.

It will be a great chance to get to see the tools that Yesplan are providing to the UK and European cultural sectors. There will also be a chance to hear of the experiences of the growing number of UK venues already harnessing the power of the system.

Production Managers, Programmers and Venue / Operational Directors as well as front line operational staff will find the content particularly relevant.

The event is free to attend and places are limited. You can register here

For more information visit the website www.yesplan.co.uk

First Speakers and Sessions for #TPC2018

Ticket Professionals Conference TPC2018

 

With just over five months until the next Ticketing Professionals Conference (TPC2018), the first speakers and sessions have been announced.

The response to the call for papers was superb, with over 65 sessions being put before Education Panel of industry professionals.

It is clear from the sessions announced so far that the focus in 2018 will be on continued improvement of revenues and customer service / interaction.

I’m sure we can all agree that these should form a core of our focus.

With more sessions and speakers to be announced we look forward to the event surpassing attendance and great feedback from delegates at the 2016 and 2017 events.

The Early bird price is still available and you can pick your registration up for the two day event from only £199+VAT here

 

 

Eventim Partners with FC United

Eventim Forms Partnership with FC United of Manchester

LONDON 2 August 2017. Eventim UK are pleased to announce their partnership with FC United of Manchester, the community football club owned and run by its members. Eventim will now deal with all of the club’s ticketing provisions including season tickets for 2017/2018 and 3-year season tickets, all available now.

 

The partnership came about after Eventim attended one of the club’s matches, being impressed by the family feel of the club they approached FC United regarding their ticketing. As FC United of Manchester matches attract crowds of more than 2,500 (several times the league average) ticketing had become a problematic area for the club.

 

Adrian Seddon, Board Member of FC United of Manchester, explains:

 

“Ticketing was an area which placed a big strain on our staff and volunteers, especially when dealing with the season ticket and membership rush over the summer and although they coped admirably this partnership will be very beneficial to the club.”

 

Eventim has extensive experience ticketing sports events, as a ticketing partner for the Rio 2016 Olympic and Paralympic Games, and as the official ticketing partner for some of the biggest names in European football, such as; Borussia Dortmund and Ajax.

 

The offering includes season ticket and membership card fulfilment, access control, print at home tickets and a co-branded ticketing webshop.

 

FC United of Manchester season tickets for 2017/2018 or a 3-year season ticket are available to purchase now on Eventim’s website.

 

Audiences’ ‘experience’ crucial in engagement?

Most arts and entertainment organisations target previous attenders to persuade them to attend a new event by selecting people on their ticketing system database who had attended events of a similar character in the past. Fairly obviously this is done by either the title of the event e.g. a play, or for a concert perhaps the repertoire, or simply “comedy” or “dance”. This is a common practice, notwithstanding the fact that human happiness comes from someone being persuaded to attend something new, different, that they then enjoyed. Perhaps we should be targeting them on opposites?
‘high definition performance’
But my point is that if we are targeting people based on what they have attended before, we need to understand the ‘experience’ they had, and use it as part of our strategy to engage people. The Observer theatre critic in the 1960s Kenneth Tynan wrote about what he called ‘high definition performance’ as transcending an audience’s usual experience. Perhaps the ‘experience’ experienced needs to be at the heart of ticketing and marketing? Bear with me:
I like a wide range of music, especially live, and while I like going to symphony orchestra concerts, chamber music and recitals, I often find the presentation of classical music in concert “dull”. The ‘no announcements, no introductions, keep the house lights up, don’t clap between movements, earnest faced musicians ignore the audience except when it claps’ presentation style, I find drains the energy out of the event. I can still sometimes really appreciate the music and the performance, but something is definitely missing. I would be worried about trying to persuade an audience to attend another “dull’ concert just because they had already attended one, though for some the music to be played is their key deciding factor.
playing from memory
This was spectacularly brought home to me when the Cambridge Summer Music Festival announced the Aurora Orchestra performing Beethoven’s Symphony No.3 in Eb major known as ‘Eroica’,to be played from memory at the West Road Concert Hall in Cambridge on 21 July. I didn’t know that playing from memory like this is rare for orchestras, and it is not known in ‘living memory’ when last it was done in the modern era. We might expect it of chamber ensembles such as string quartets and even some choirs, but not orchestras.
West Road is a modern unassuming concert hall, part of the university though out of term time, with a bright clear acoustic, a single rake of seats and open end concert platform. As the audience gathered, the stage had a handful of music stands and small platforms, no instruments, and the musicians did not appear on stage to prepare for the appointed start.
clearly choreographed
Instead Tom Service, BBC Radio 3 presenter bounded across the stage, microphone in hand, talking rapidly about the concert to come, quickly followed by conductor Nicholas Collon and the musicians carrying their instruments, though not heading for any obvious playing positions. Instead , in a double act, Tom and Nicholas, talked about how the symphony came to be written, its’ structure and themes, with the musicians moving to form groups and playing extracts, mostly staying standing up.
This was clearly choreographed, at one stage the musicians forming a semi-circle around the platform and Nicholas “surfing” the tune around the different instruments. This was clearly also immensely entertaining to audience and musicians, with some participation as we were asked to help sing a tune, and at another point the players stamped out the rhythm. The energy and enjoyment was palpable.
the physicality of playing standing
First the Aurora Orchestra was to play Richard Strauss’ Metamorphosen (which includes a quote of a theme in the Erocia), for 21 string soloists, so the musicians left the stage and two stage crew rapidly set up, including music stands, before they returned, the majority standing, to play. This in itself was an astonishing experience, the orchestra exciting the emotions from the work, delivering the vigour and the elegies of the piece, definitely gaining from being mostly standing, and the physicality of playing standing, knees bending to the rhythms, heads bobbing, arms vigorously sawing bows. The inter-action with the conductor, between the players, with us, was tangible. This got loud long applause.
After the interval, the orchestra returned, again mostly standing, no music stands, and great smiles of anticipation – that can’t be faked. They then played the Eroica, the music soaring and roaring, suddenly the themes and tunes, identified in the introduction, excitingly audible, and somehow visible in the performance of the players. The sheer energy yet control of the performance was remarkable, and the orchestra as a living breathing whole, inter-acting together, smiling and nodding at each other, and sharing with us, quite remarkable to witness. This enriched the performance and made it palpably tangible and visceral, a shared experience of the music. Actually, it is almost impossible to describe.
As the last note hung in the air, the audience now roared and stood as one for the loudest standing ovation I have witnessed (remember this is Cambridge, university town in England). After the applause died down and Nicholas Collon eventually left the stage, the musicians hugged and kissed each other!
what has it to do with ticketing?
Now dear reader, why have I explained all that and what has it to do with ticketing? Because it is not about what that audience attended, but about their experience of it, which in my own lifetime’s experience was unique and exceptional. And the ticketing database needs to be able to record for those attenders that this was an exceptional experience.
take the audiences experiences into account
Sometimes it is easy to connect what was seen with the detail of the experience – some musicals for example – but sometimes it is now and again in the run of a play or an opera, or dance, and certainly so with music. It may be exceptional to that audience in that venue, but it needs a flag and a narrative description in the attenders’ records. And when thinking about targeting, we should start taking the audiences experiences into account.
When an audience has had such an experience, it is my view that we are definitely further down the road to engaging with them, and we can quietly remind them that they shared a remarkable experience when persuading them to attend something else.
The Aurora Orchestra played the same concert for the BBC Proms on Saturday 22nd July 2017 broadcast on Radio 3, so catch it on iPlayer (but radio remember), and the Aurora make their Concertgebouw debut with this in Amsterdam on Friday 4 August 2017 at 8.00pm: http://www.auroraorchestra.com/event/eroica-from-memory/ 

AudienceView Acquires TheaterMania and OvationTix

TORONTO, Ontario, Canada – July 11, 2017 – AudienceView, a world leader in e-commerce software for events and entertainment organizations, is pleased to announce the acquisition of TheaterMania, including the OvationTixTM Software as a Service product as well as the TheaterMania.com and WhatsOnStage media brands. As part of AudienceView’s continued investment in its market-leading position providing ticketing, CRM, and fundraising solutions, the acquisition of TheaterMania extends AudienceView’s market reach to more than 2,000 arts and culture, sports, live events and education organizations globally, from the largest entertainment groups in the world to single weekend festival events.

 

“AudienceView and TheaterMania have long shared a common vision and passion to help entertainment organizations build devoted communities and fulfill their missions,” said Gretchen Shugart, formerly CEO of TheaterMania and now President, Arts and Culture of AudienceView. “We are truly thrilled to be aligned with an organization that understands the industry that we serve and has aggressive plans to invest in our products and offerings to drive even more success for our clients.”

 

“The combination of AudienceView and TheaterMania now provides arts and cultural organizations with best-in-class capabilities to control their brand and business operations while tapping into the immense power that effective distribution channels bring,” said Mark Fowlie, CEO of AudienceView.  “This acquisition expands our portfolio to become the ideal destination for organizations of all sizes seeking the best technology, services, and partnerships to drive their businesses forward.”

 

In keeping with the company’s unwavering commitment to customer success, AudienceView is dedicating additional investment to be focused on providing superior client service and support. The first-class OvationTix service and support will continue without change and will benefit from additional support, resources and expertise from AudienceView.

 

Additionally, the company will be bolstering product investment in both the OvationTix and AudienceView platforms and will be building innovative solutions that will be shared across both offerings.  Further, AudienceView’s customers will benefit from the significant audience reach, event listings, and multimedia content provided by TheaterMania.com and WhatsOnStage.

 

“Whether a venue has an audience of 99 or over 100,000, AudienceView is now the most compelling choice for organizations that want to grow their communities through innovative technology, strategic distribution strategies, and a team of experts dedicated to creating and supporting customer success every single day,” adds Mark Fowlie.

 

AudienceView will serve its customers from its Toronto and London offices as well as the New York TheaterMania offices.

 

JEGI served as the exclusive financial advisor to TheaterMania.

 

To learn more about AudienceView’s acquisition of TheaterMania and OvationTix, please visit: http://bit.ly/tm-ot-acquisition.

 

Breaking Down Complaint Letters

Quite a lot of projects this year have been focused on service and expectations our customers have of our ticketing operations, either as people, products or digital services.
We all know things go wrong, people (our staff) have bad days or even customers being a little bit on the grumpy side. Whoever is at ‘fault’ for the issue, however ‘big’ the effect was on the activity, you can rest assured that all complaints, well pretty much all complaints have the same basis make up.
1. About them and why you should listen to them
All very simple to start off and I guess it’s good to give the recipient of your complaint some idea about who you are and that you are experienced and know how this stuff should work.
Typically a phrase such as “I have been coming to XYZ for over 20 years” or occasionally the non numeric version, “as a regular attender over recent year”
highlight persona or personal circumstance, not membership status or longevity of patronage as flag to follow up
I think these set the scene really well, as when reading complaints (or praise) we must appreciate the persona of the author, if a ‘regular’ attender cannot find their way to the bar or park – we must think about our new customers – as they will almost certainly have an issue.  We should not however use the about me to prioritise the email, it is from one person, albeit a group of people may have been in the booking, it is not from £5,610 of revenue, so when looking through the letter highlight persona or personal circumstance, not membership status or longevity of patronage as flag to follow up
2. The Actual Issue
It is very often the case that the ACTUAL issue can take up a few short sentences of a multi page customer service rant. In the ticketed environments we work in, most AI’s fall Into one of five pretty clear categories
  • The seats were crap / overpriced ( we are talking restricted view, leg room, general pricing, booking fees, someone was talking in the row behind, etc)
  • My buying experience was crap (people were rude to me or website did not work/ I couldn’t use it)
  • I did not get tickets, everyone else did, your system sucks (pretty simple really, but when is a system ‘fair’ if you miss out?)
  • I don’t agree with your published policies and conditions (or did not read them – often combined with #3 ‘as a hard working junior doctor etc)
  • I work in IT / Customer service (catch all phrase – basically telling you what you are doing wrong and why they know better)
Now there is some tongue-in-cheek in those five, but they ARE the issues, we need to focus on these more than anything. We only have these scant view lines to work out where (if anywhere ) we have let the customer down or ‘could have done better’
3. What Makes you so different ( AKA the ‘multipliers’)
Depending at the entry point to your complaint above, the customer will deploy varying degrees of ‘multiplier’ to their story or issue to swell its impact. Now before we start, we must understand that for many people a trip to the theatre IS a special occasion, may reading this may go to three shows a week / month / quarter, but many of our customers may go to one or just a few a year. Invariably these is around a special occasion, typically birthday or anniversary.
Of course for every celebration there are the darker multiples, ‘her husband died last year’ or ‘I had an ingrown toenail removed, so could not possibly stand’ (genuine multiplier)
Celebration or a darker anniversary or circumstance, we do need to park the multipliers to one side when evaluating a complaint. This may sound harsh, but if we weight the story and not the issue, we risk focusing on circumstance and not resolution.
I was in the closing arguments of a murder trial I was defending, I could hardly ask the judge for an adjournment to come and get tickets
There are then the multipliers that are based on work, some of these are my favourite. It tends to be used by ‘professionals’, more than management, skilled or unskilled workers, as if we should pay more attention to the complaint of a barrister to that of a barrista – I have seen complaints using phrases such as ‘I was in the closing arguments of a murder trial I was defending, I could hardly ask the judge for an adjournment to come and get tickets’. Let’s think about that. No, they could not have done that, but when that barrister was on her way to court, if she had not been able to get a de-caf mocha choca-cino as the barista at Cafe Nero WAS queueing up for tickets all hell would break lose.
We should pay attention to the personal story, if only to have empathy and to be able to understand why they we’re disappointed, it should not, like personal circumstance provoke more or less of a reaction to the underlying issue.
4. The Claim
I would like you to rectify the situation – eight simple words. They show I am rational and think you should be given a chance to apologise (number 1 thing you should do anyhow) along with refund, exchange, making tickets available that could not be obtained.
The customer does not have to leave it up to you to decide on the response in service or monies that you may give. A nicely put, ‘I request a full refund for the seats I was unable to use’ is perhaps a justified request.
5. The Threat
So, if you do not give me what I want, I am going to do ……… which is normally, a #1 + #3 reversal – so stop being a member / donor, head to your competitor or raise a complaint with trade / consumer body.
 The threat should be the least of your worries, really!
If it is something you or your staff have genuinely done wrong and let down a customer in either service or product, you will want to correct it, right? Of Course you will. The threat tends to be used when the complainant knows that they are fault, have no or little basis for their counter claim. This can be seen as a form or bullying, expoesinally when tied at junior or less experienced staff.
6. The Wrap Up
So, we’ve heard about them, the issue, why they are so different, what they want and what they’ll do / no do if you don’t react, what else could be added? Well, that’s the ‘Wrap Up’. It’s easy to read behind the lines here, with themes of you’re useless at your job and everyone hates you, along of course with the challenge of “I know you don’t care about your customers, so you probably won’t even reply” – this is also know as the ‘Shitty Wrap Up’
Wrap ups are just or should be just that. We know the customer is really saying ‘I feel really let down and hope you can see why, please alter your process and / or (not always) refund me something. If the customer has written this, take it on board. If it’s a SWU – ignore that part and make your own from the actual problem and the claim, here the problem and consider the solution.
Finally
Not ALL complaint letters follow this exact structure but a lot do, or at least a number of elements within them. We must remember that the product we supply is associated with many other emotions, celebrations and traditions, we are offering far more than a parcel from Amazon, so when things go wrong the perceived damage or sense of injustice grow.
Next time you are reading a complaint letter, take a highlighter, highlight the issue and answer that, referencing the about me sections, but ignoring the multipliers. The threat and wrap are almost worth redaction completely, they add noise to what are focus should be – just solving problems and making customers have a wonderful event.

What are we counting?

Roger is not going quietly, and here is his latest, originally published by Arts Professional here: https://www.artsprofessional.co.uk/magazine/article/what-are-we-counting

This version is updated.  Roger has more questions than answers about the quality metrics system that Arts Council England’s larger NPOs will soon be required to use.  He was surprised to find the Arts Council of England’s only response to the original version of the article was to query his comments about the ACE contractors Culture Counts.  Researchers at Flinders University in Adelaide have now published an article in Cultural Trends ‘Counting culture to death’ severely criticising the Quality Metrics concept.  Liz Hill reports in Arts Professional on that here: https://www.artsprofessional.co.uk/news/only-fool-or-knave-trusts-quality-metrics-say-academics

Counting what counts?

People keep asking me what I think about quality metrics, the audience research system that Arts Council England (ACE) will shortly require its largest National Portfolio Organisations (NPOs) to use.

When I try to answer this complex question, many immediately tell me they were asking confidentially and don’t want their own views known. I hear a lot of reservations and many worries, but everyone seems reluctant to say anything during the current NPO application process.

Whilst understandable, this is not helpful. It is surely essential to embark on a proper discussion of whether this will deliver reliable results for NPOs and ACE, and to address people’s concerns.

Uneasy questions

I have been a champion of audience data for a long time. I conducted my first year-long audience survey at the Vic in Stoke on Trent in 1969, supervised by Keele University. I have been commissioning research surveys for over 40 years and the Arts Council published my book ‘Boxing Clever’ on turning data into audiences in 1993. And I have collaborated with them on many audience initiatives, including the drive to place socio-economic profiling tools at their NPOs’ fingertips.

So, I ought to be welcoming the concept of quality metrics and what Culture Counts proposes to deliver for Arts Council England. I can see why Marcus Romer (read his blog from 27 September) would welcome the voice of the audience, as end-recipient of the art, into ACE thinking. But I am left with a lot of uneasy questions, mostly methodological.

Unreliable research

Most people with any knowledge of research methodology are asking the same questions, because this type of research is inherently unreliable, yet a lot of reliance is being placed on the findings.  The recent experience of surveying potential voters in the polls prior to the UK election on 8 June have rather confirmed the unreliability.

The Arts Council’s own former Senior Marketing Officer, Peter Verwey, constantly reminded arts marketers of the inherent unreliability of audience surveys, unless there were controls to manage the sample. Even then, reliability depends on respondents understanding the questions. If you ask a question and the respondent can’t ask for clarification on what the question means, then the answers can’t be relied upon. But if explanations are given, then bias creeps in, depending on what is said to them.

Sadly for Welsh National Opera, the majority who said when and where they had seen an opera, turned out to not actually have attended an opera at all.

At the Arts Council of Wales, we used Beaufort Research to check respondents’ understanding of some simple questions about the arts, including: “When did you last attend an opera?” Sadly for Welsh National Opera, the majority who said when and where they had seen an opera, turned out to not actually have attended an opera at all. The public have a very different understanding of the words we use to discuss the arts, and this can have a significant impact on whether survey questions are completed.

This is an inevitable drawback of quantitative research. Researchers have to decide in advance what precise questions to ask and have to constrain answers to a fixed choice. Qualitative write-in answers can’t produce reliable, comparable results, even though narrative answers can provide the richest source of our understanding of what a specific audience member thought.

Biased responses

Audience surveys have other equally large flaws. Peter Verwey’s joke was that the survey samples usually comprised “anyone who had a working pen/pencil when the survey was handed out”, though that has presumably changed to whether people have an email address and bother to open survey emails.

Surveys conducted in foyers after performances are inherently biased in that they capture only those with time to answer. And even “there is an app for that” only suits the tech savvy.

Analysis over the years shows that completion is biased in favour of the most supportive members of the audience and those keen to make their views known, sometimes complainants. You can overcome some of this by ruthless random sampling – only looking at the feet of the people to be selected to answer the questionnaire, for example – and similar techniques can be applied online. But the bias, of who actually responds when invited to, remains.

These days, when we are capable of creating a socio-economic profile of attenders who book tickets, we ought to, as a minimum, be expecting the quality metrics methodology to include a check for the representativeness of the sample.

Incomparable performances

Perhaps the biggest challenge is that audience surveys are inherently situational. They can only reflect what happened in a particular venue on a specific date and time, and gather the opinions of the people who both attended and chose to respond. If you have ever been a house manager and experienced the difference that a large group booking can make in an audience, you will understand the potential variability.

This makes comparability from one performance – let alone event – to another very difficult. Researchers have known about these issues for decades and therefore attempts to measure or assess impact based on audience surveys are always approached with huge caution, even if conducted for a single venue or a single performance.

Unexplored impact

Arts Council England’s own 2014 literature review by WolfBrown was clear about this: “the literature raises questions as to the plausibility of aggregating survey data across organisations and artforms, due to the highly personal and situational nature of impact, and because of differences across the forms themselves.”

It can be argued that valuing art based primarily on the experiences it produces, in fact devalues the work itself. Can you really tick a box to encompass your opinion? Indeed, post-event surveys primarily measure the ‘experienced impacts’, perhaps within a day or so, and ignore the ‘extended impacts’, probably weeks or even years later (typically assessed through retrospective interviewing and longitudinal tracking studies).

And while we try to understand these impacts on each individual, what role did pre-attendance marketing, the venue, pre-show talks, the people who attended with them, and the rest of the audience, have on the experience? Some researchers have expressed serious concerns about comparing self-reported audience experiences across different artforms and contexts because of the huge range of impossible-to-control variables being measured in these, in effect, crowd-sourced reviews.

Flawed evaluation

I had expected recent reports, commissioned by ACE, to provide the answers. I was surprised to find the final report on the quality metrics national test was assessed and written by two staff from the company that ran the pilot scheme, John Knell and Alison Whitaker. So the researchers were being asked to mark their own homework. Highly unusual, regardless of their integrity.

The Arts Council did commission an independent evaluation, from Nordicity (though the researcher is not named) though this only examined the experience of the organisations participating in the National Test, and not the methodology used in the pilot or the internal processing of the resultant data.

serious concerns raised by participant organisations about the methodology

Nonetheless, that evaluation reported some serious concerns raised by participant organisations about the methodology, saying “the majority of consultees questioned the reliability of the resulting data because of the sample frame, in terms of its representation and size” and commenting that “this aspect evidently impacted the organisations’ use of the data, with organisations unconfident to draw any firm conclusions, unable to ‘convince’ programmers of its value, and unsure of what ‘robust’ would look like in practice.” It went on to say “consultees suggested a number of areas where unintended bias or skewed data had the potential to be introduced. It is evident that these elements contributed to consultees’ overall opinion that the resulting data did not accurately reflect the quality of their work.”

Knell and Whitaker’s report makes no reference to statistical significance or reliability, or the representativeness of audiences; and despite references to “highly sensitive aggregation” there is no explanation of the basis for that data aggregation, except for some crude geo-location, artform, gender-based data-merging. It’s impossible to discern how they have overcome the huge problems of situational audience surveys and event comparability.

There is also no explanation of how audience responses are related to the other elements of the triangulated quality metrics research process, namely peer responses and internal assessments. Neither is there an indication of how respondents were selected. Indeed, the report is more about the findings of the surveys than testing the reliability of the methodology or its underlying fitness-for-purpose and statistical reliability.

Sampling problems

Obviously it is easiest to select survey respondents from ticket bookers with email addresses, and some of the organisations that participated in the pilot research chose people with particular characteristics, or a certain frequency of attendance. Some indicated that they wanted to input the data findings into their CRM systems. Did they select target samples accordingly?

Some added extra questions of their own to the survey, which in themselves might have affected understanding, response rates and completion. There is no explanation of how these additional questions were tested for respondent understanding. Also, only ‘30 responses’ is cited as an acceptable minimum for an event to be evaluated. How does this relate to the total attendance? There is no rationale given for this low number and no indication how an event with 30 survey respondents will be compared with an event with 300.

What’s more, there is no indication of how any of this will be possible under the new General Data Protection Regulation and its specific granular consent regime, which could further reduce the number of attenders available for survey and the use and processing of their responses.

The better news is that over 19,000 surveys were completed in the national test. This is clearly a large sample in UK terms, but size is not enough, especially when the integrity of the sample is unclear. We can’t rely on the national sample size if we need local reliability. We need to understand the reliability of the findings for each individual organisation in their unique catchment area. And we need to know the profile of the survey respondents in the context of both the universe of NPO attenders, and the actual attenders at each individual organisation.

Finally, there are of course other providers of post-attendance survey tools, and arts organisations already carrying out frequent surveys of attenders are worried about wear-out from over-surveying core attenders. All their other surveys are intended to understand audiences better and guide marketing, operational and audience development issues, not inform critical ACE grant award decisions.

ACE has a lot of questions to answer

I write this because I find ACE has a lot of questions to answer if it is to reassure arts organisations about the methodology and the quality of its proposed metrics. Just what is it counting, and exactly how?  The researchers at Flinders University in Adelaide have confirmed the academic concerns about using such research methods.  They write that metrics-based approaches to assessing cultural value “invite political manipulation and demand time, money and attention from cultural organisations without proven benefit”.

Their paper, ‘Counting culture to death’, refutes the “widely held belief” that “a set of numbers can provide vindication, or at least insurance, in the constant struggle to justify public funding”. They conclude that attempts to quantify cultural value are not delivering on their promises, and bring “destructive” unintended consequences.

The paper states that using indicators and benchmarks to assess cultural activities, “which exhibit no obvious capacity for scalar measurement” is a “political act”. The “ostensible neutrality” of this approach is, they say, “a trick of the light trying to launder responsibility for judgment in the competition for scarce resources”.  It is certainly clear to me that relying on an unreliable methodology could have dangerous consequences for the Arts Council.

This article is Rogers personal opinion and does not reflect the views of his colleagues or any other organisations.

Troy Kirby Podcast interview with Roger Tomlinson

Roger is not going quietly:

Troy Kirby of the renowned SportsTao in the US followed up on Roger’s ‘Getting Permission Wrong?’ piece by interviewing him for his regular podcast – up to number 775 – and the two had a conversation which ranges over management use of data, attitudes to ticketing and the Box Office, customer relations, ‘big data’, communications, sales teams, and even pricing, discounting and comps.

http://troykirby.com/ep-775-roger-tomlinson-uk-ticketing-consultant?tdest_id=321128