Herding cats and counting sheep

Whilst away Church of England attendance figures for the year were released, triggering the usual ritual Groundhog Day for terminal decline, renaissance, or business as usual, depending almost entirely on the prejudices and agendas of the people doing the prognosticating. It doesn't seem to matter much what they actually say. Everybody, of course, is abslutely right from their own point of view, especially when it’s buttressed by hard figures.

As it happens, however, yesterday Kevin Ryan, diocesan ICT manager came yesterday to explain to Bucks area deans and lay chairs the use of Tableau, a new statistical tool we are making available for deanery planning. This enables people to drill down easily into local statitsics. As well as Kevin, we were lucky that one of our lay chairs is a professional in the field of marketing statistics, and was able to correct many basic errors of perception and interpretation among our group of 21 people. From this exercise we learn a few home truths:

1. The interpretation of statistics for serious marketing purposes is a fne art. What you are looking for affects what you find, and all kinds of allowances have to be made, with local knowledge, before you can arrive at a real answer to pretty much any particular question. It can be done, but not without a lot of work and qualification. Look at two examples from Bucks of explosive growth, more than doubling numbers over five years. One is caused by the fact that it is a reopened former redundant church near new housing.The other is caused by significant Evanglism more than doubling the size of an already large congregsation in five years. Make what you will of either, but to make sense of data you need to consider contextual and local realities.
   
   
2. Figures usually look enticingly accurate and absolute simply because they are figures, and only very occasionally because of the precision with which they describe particular realities. Go to any school governors' meeting in the league tables season, and you'll see what I mean.
   
   
3. Everything depends on the accuracy of the data that went in to begin with. Church statistics, especially attendance ones go through three significantly wobbly layers of filtration which cannot be avoided... starting with the fact that they are collected largely by volunteers.
   
   
4. Level 1: One is the significant differences in counting methods and consistency, which vary from never to regular, as well as from scrupulous to megasloppy. When I was an incumbent I once took a Sunday out to check our numbers. I physically counted 242 people on the premises. The number in the book was 127. This led to two significant discoveries about my parish. (a) the greatest variable factor was usually, in fact, which verger had done the counting. The more otpimistic ones tended to count higher, the more pessimistic lower, and some always the same kind of figure by a process of dead reckoning whereby they would go to the vestry, check the figure they'd put in last time, ask themselves whether the service felt fuller or not this week, and then stick something in the book. I’m not proud of this doscovery, but it was the way I discovered things were generally done. Accuracy was higher for smaller services simply because it was very much easier to count 16 people at evensong than 242 at the main morning service.
   
   Level 2: To derive the usual Sunday attendance figures, you need to average four Sundays in October noting any exceptions (like, say a baptism in the main service). This, along with the mathematical abilities of clergy and wardens, reveals a cornucopia of results, some more compliant with the theory of how they should be collected than others.
   
   Level 3: Returns rates vary. Only one deanery out of ten had an almost complete dataset and for another it was almost non-existent. Those who want to count sheep need first to herd cats.
   
   
5. As a result of all these realities, the grossed up statistical picture may, surprisingly, be the most “accurate.” I don’t mean the figures are actually correct, they surely aren’t, but grossed out across a field through a few years, trends may well appear, with the inaccuracies cancelling each other out a bit. The big story seems to be one of significant and fundamental erosion in the nineties, and a generally flatlining pattern this century. It’s a good way of disappointing everyone with an axe to grind, but it seems to be the truth. It’s is probably the least promising straw from which to make bricks to load into coshes, but I'm sure noble attempts were made last week.
   
   
6. The best use of these figures is local, because that's where people have the contextual knowledge to make sense of them. Thus our use of Tableau.
   
   
7. These figures are like reported crime rates. Other oft-quoted data, for example, the English Church Census, is of the nature of the Home Office Crime Survey — the dataset behind it is very much more partial and any extrapolation from it very much more an expert game, like long range weather forecasting. Caveat Emptor.
   
   
8. The figures we drilled down into, with some degree of local knowledge, revealed a huge variation, from the kind of classic decline patterns Fleet Street assumes, which do exist occasionally, to a far more egneral static pattern, and one or two examples of swift decline or explosive growth. More growth than devcline among these last. There are a handful of Bucks Churches which have experienced the kind of growth where they have had to double up shifts to fit everybody in, and one deanery in which, across almost thirty parishes, numbers have more than doubled over the past five years. What are you likely to experience in any of our 288 places of worship in Bucks? If you really want to know you’ll just have to go and see...