Opera in cinema: a fresh look

English Touring Opera recently released the “results” of a survey they had run about audiences’ attitudes to opera in cinema. The Stage headlined it as “Opera screenings failing to boost interest in the art form, survey finds” and this was the gist of most of the reporting of the story.

I’ve had a good look at the survey report and, unsurprisingly, the headlines don’t necessarily reflect what’s there. Firstly, it’s a well carried out study, with robust methodology, although there are problems with the sample. It’s both very small (234 for the quantitative part) and almost completely London-based. The findings might have been very different if it had been carried out regionally.

Statistical Insignificance

I’ve just run a split test.  After 8 days, Kissmetrics Statistical Significance Calculator tells me “We are 100% certain that the changes in Test B will improve your conversion rate.”  In fact, I ran four tests and got this response for two of them. The graphs show a solid increase in each case (a 28% improvement in one and 42% in the other).

Clearly, for the two that worked, I should switch to the new version and for the other two go back to the drawing board.  I did something right for two of the tests and didn’t quite get the creative right on the other two.

Except….

Clarification on A/B Testing

To clarify my previous post, I’m not against A/B Testing when it’s carried out properly, and I’m certainly not against experimentation. I am steeped in the idea of continuous improvement and think you should be continually seeking to improve your product and its promotion.

A/B Testing often shows a difference but does it make a difference?

Which of the following statements are true?

1) A/B testing is at best a superstition and at worst a scam (ok, let’s be kind and say it’s a bandwagon that digital marketers are happy to ride)

2) A/B tests are attempts to do a complicated measurement using a simple tool and are not fit for purpose

3) With an A/B test you are only comparing two data points and not looking at any data outside of the test

Am I A Marketing Genius? You Decide

Following my previous posting, in which I said marketers are quick to attribute results to their own activity, I’ve just had a small but perfectly formed example.

For some shows I’m selling in London, I made a change last week (basically switched ticket agents in order to reduce booking fees for the customer) and I’ve been anxiously watching sales all week to see if it had an effect.

At the end of the first week, ticket sales through those channels increased by 70% on the average for the previous 6 weeks, with a 39% increase on the highest of those earlier weeks (always beware averages!). And all this in a week when the sun came out and people deserted theatres and cinemas in droves.

Correlation or Causality? Why Asking Why is Important

This amused me:

http://www.wired.com/epicenter/2012/05/the-ab-test-results-are-in/

I like the bit that says “Don’t question why, just make the change.”

The claim is that the movement of a button on a website by a couple of inches caused a 264,000% (sic) increase in clicks.

Oh God, not Samantha Brick AGAIN!!! Masterly Manipulation or Simple Serendipity?

What on earth am I doing pitching in to the over-blown, done-to-death Samantha Brick/Daily Mail furore? Surely there’s NOTHING else to be said?

Well, like millions of others, I was fascinated by this story. It had something to irritate and appall almost everyone: there was gender politics, a dash of titillation and lashings of bad behaviour, from delusional vanity and sisterly betrayal to vicious cyber bullying. If the original pitch didn’t float your boat, there was something for everyone in the various spin-off debates.

Why does nobody in the theatre look at ROI when committing marketing spend?

I don’t know if it’s declining educational standards these days but nobody in theatre marketing seems to do their sums (apologies if you’re one of the 2 or 3 who actually own a calculator). The nearest I ever get to an ROI calculation is “well, it went OK the last time time we did it.”

I have an ongoing argument with a certain rather large theatre chain (no prizes for guessing which) who insist on selling email addresses at 3p each, which is £30 per thousand. That might be alright if they were a commercial email vendor and the buyer was using them to sell high price items but it doesn’t work for a business partner on a tight budget.

The Dangers of A/B Testing

The following is not for people who are experienced in A/B Testing and who carry this technique out regularly and professionally. Rather it is a word of warning for people who are maybe thinking of trying it for the first time and are tempted to take a superficial view of what it will do for them.

What is A/B Testing? Very simply, it is a means of testing different creatives – copy, images, calls to action – by trying different options and then going with the one that gives you the best results. It has become a widely accepted tool for digital marketers.

20 Things I Have Done During 2011

1. Blogged intermittently
2. Completed a CAM Diploma in Digital Marketing
3. Learned a lot about Digital Marketing and myself
4. Decided to do a Web Analytics and Social Media Monitoring module next year
5. Completed my first year of CPD with the CIM
6. Launched a new website
7. Stopped tweeting (except for business)
8. Not used Facebook (except for business)
9. Used Twitter lots for business
10. Used Facebook lots for business
11. Stopped writing opera and theatre reviews
12. Got an apptastic iphone
13. Run lots of Facebook and Google Ad campaigns
14. Filmed a number of videos none of which have gone viral
15. Struggled with an email database that’s going downhill fast
16. Not sorted my Youtube channel out (on the list)
17. Eaten approximately 700 bananas
18. Got hooked on Made in Chelsea (oh yes)
19. Had shockingly bad experiences with Groupon and Wowcher
20. Sold about 40,000 tickets