Herding cats and counting sheep

Whilst away Church of England attendance figures for the year were released, triggering the usual ritual Groundhog Day for terminal decline, renaissance, or business as usual, depending almost entirely on the prejudices and agendas of the people doing the prognosticating. It doesn't seem to matter much what they actually say. Everybody, of course, is abslutely right from their own point of view, especially when it’s buttressed by hard figures.

As it happens, however, yesterday Kevin Ryan, diocesan ICT manager came yesterday to explain to Bucks area deans and lay chairs the use of Tableau, a new statistical tool we are making available for deanery planning. This enables people to drill down easily into local statitsics. As well as Kevin, we were lucky that one of our lay chairs is a professional in the field of marketing statistics, and was able to correct many basic errors of perception and interpretation among our group of 21 people. From this exercise we learn a few home truths:

1. The interpretation of statistics for serious marketing purposes is a fne art. What you are looking for affects what you find, and all kinds of allowances have to be made, with local knowledge, before you can arrive at a real answer to pretty much any particular question. It can be done, but not without a lot of work and qualification. Look at two examples from Bucks of explosive growth, more than doubling numbers over five years. One is caused by the fact that it is a reopened former redundant church near new housing.The other is caused by significant Evanglism more than doubling the size of an already large congregsation in five years. Make what you will of either, but to make sense of data you need to consider contextual and local realities.
   
   
2. Figures usually look enticingly accurate and absolute simply because they are figures, and only very occasionally because of the precision with which they describe particular realities. Go to any school governors' meeting in the league tables season, and you'll see what I mean.
   
   
3. Everything depends on the accuracy of the data that went in to begin with. Church statistics, especially attendance ones go through three significantly wobbly layers of filtration which cannot be avoided... starting with the fact that they are collected largely by volunteers.
   
   
4. Level 1: One is the significant differences in counting methods and consistency, which vary from never to regular, as well as from scrupulous to megasloppy. When I was an incumbent I once took a Sunday out to check our numbers. I physically counted 242 people on the premises. The number in the book was 127. This led to two significant discoveries about my parish. (a) the greatest variable factor was usually, in fact, which verger had done the counting. The more otpimistic ones tended to count higher, the more pessimistic lower, and some always the same kind of figure by a process of dead reckoning whereby they would go to the vestry, check the figure they'd put in last time, ask themselves whether the service felt fuller or not this week, and then stick something in the book. I’m not proud of this doscovery, but it was the way I discovered things were generally done. Accuracy was higher for smaller services simply because it was very much easier to count 16 people at evensong than 242 at the main morning service.
   
   Level 2: To derive the usual Sunday attendance figures, you need to average four Sundays in October noting any exceptions (like, say a baptism in the main service). This, along with the mathematical abilities of clergy and wardens, reveals a cornucopia of results, some more compliant with the theory of how they should be collected than others.
   
   Level 3: Returns rates vary. Only one deanery out of ten had an almost complete dataset and for another it was almost non-existent. Those who want to count sheep need first to herd cats.
   
   
5. As a result of all these realities, the grossed up statistical picture may, surprisingly, be the most “accurate.” I don’t mean the figures are actually correct, they surely aren’t, but grossed out across a field through a few years, trends may well appear, with the inaccuracies cancelling each other out a bit. The big story seems to be one of significant and fundamental erosion in the nineties, and a generally flatlining pattern this century. It’s a good way of disappointing everyone with an axe to grind, but it seems to be the truth. It’s is probably the least promising straw from which to make bricks to load into coshes, but I'm sure noble attempts were made last week.
   
   
6. The best use of these figures is local, because that's where people have the contextual knowledge to make sense of them. Thus our use of Tableau.
   
   
7. These figures are like reported crime rates. Other oft-quoted data, for example, the English Church Census, is of the nature of the Home Office Crime Survey — the dataset behind it is very much more partial and any extrapolation from it very much more an expert game, like long range weather forecasting. Caveat Emptor.
   
   
8. The figures we drilled down into, with some degree of local knowledge, revealed a huge variation, from the kind of classic decline patterns Fleet Street assumes, which do exist occasionally, to a far more egneral static pattern, and one or two examples of swift decline or explosive growth. More growth than devcline among these last. There are a handful of Bucks Churches which have experienced the kind of growth where they have had to double up shifts to fit everybody in, and one deanery in which, across almost thirty parishes, numbers have more than doubled over the past five years. What are you likely to experience in any of our 288 places of worship in Bucks? If you really want to know you’ll just have to go and see...
   

Taking a Global View

Interesting times, reviewing with archdeaconry colleagues basic stats for Bucks parishes, 2003-8. The figures are wildly variable, depending on who returned forms in any year and who didn’t. No surprises there, then.

The bad news is that flakiness about data collection makes it much less useful than it could be. The good news is that most places in which people talk decline are holding their own, or exhibiting steady organic growth. There are some clear examples of decline as well, along with one or two growth hotspots.

Interesting ranges like “small rural churches” and “Slough” exhibited surprisingly steady and heathy patterns across the range, of which the parishes themselves are probably entirely unaware.

So, we all need to get the act together about figures. The message for now is that the truth out there is probably more varied and less spooky than people may feel. Fleet Street don’t know Diddly-Squat.

The task of interpreting global statistics, indeed the very thought that bishops or archdeacons might ever have to look at and interpret such culturally alien things as mere figures, puts me in mind of a marvellous poem by May Swenson (1913-1989), first published in 1963:

Southbound On The Freeway
A tourist came in from Orbitville,
parked in the air, and said:

The creatures of this star
are made of metal and glass.

Through the transparent parts
you can see their guts.

Their feet are round and roll
on diagrams — or long

measuring tapes, dark
with white lines.

They have four eyes.
The two in the back are red.

Sometimes you can see a five-eyed
one, with a red eye turning

on the top of his head.
He must be special —

the others respect him,
and go slow,

when he passes, winding,
among them from behind.

They all hiss as they glide,
like inches, down the marked

tapes. Those soft shapes,
shadowy inside

the hard bodies — are they
their guts or their brains?

Taking the Long View?

Panning in and out can make things look different. Take, for example, current fear that we are saddling our children and grandchildren with public debt on an unprecedented scale. Here’s a graph from the Telegraph, entirely accurate, portraying the problem of balooning public debt as a percentage of GDP: Dire times, you may say, and a real problem for us all.

But, you may wonder, what about the bigger picture? As public sector requirement has risen down the years, we’ve always funded it by borrowing. We never did, for example, save up in a Jam Jar until the nation could afford, say, an army and navy for the Napoleonic Wars. How does our present apocalyptic orgy of public debt sit within a historical range?

OK. Here is UK Public Debt as a percentage of GDP going back to, er, 1700. Hmm. Lies, damned lies and statistics, you may say. But it does indicate that statistics need to be placed in context.
By all means let’s not saddle our grandchildren with unnecessary debt — but let’s try and do so on rational grounds.