From Misrepresentation to Misrepresentation

The Church Times of 24 April includes a letter from Bev Botting (Head of Research and Statistics for the Archbishops' Council) and Kevin Norris (Senior Strategy Officer for the Church Commissioners), responding to my paper From Delusion to Reality, in which I critique From Anecdote to Evidence, the summary report of the Church Growth Research Programme.

The letter includes the following paragraph:

‘Dr Hart notes that the factors associated with growth account for only a small proportion of the difference between growing and declining churches (up to 25 per cent). From Anecdote to Evidence explicitly recognised this. Indeed, a quotation from one of the researchers making this point was on the back cover of the report.’

The image above is from the back cover of the report and shows the quotation to which they refer. It is flatly untrue that this quotation makes the point that the factors associated with growth account for only a small proportion of the difference between growing and declining churches. It beggars belief that Bev Botting & Kevin Norris are prepared to claim otherwise.

I challenge anyone to attempt to explain how this quotation makes the point they say it makes. And I challenge anyone to find any other reference to this point in From Anecdote to Evidence.

The truth is that this highly significant point was not included in From Anecdote to Evidence, consistent with the systematic misrepresentation of the underlying research which runs through the report.

Early in the letter, caveats included in From Anecdote to Evidence are quoted, followed by the statement that ‘It is simply not correct, therefore, to suggest that the summary report failed to include any caveats’. It would make sense to make this point if I had claimed that the report fails to include any caveat, but I never did made this charge; rather, I quote their caveats in my paper. This particular logical fallacy, called a straw man, is where you misrepresent someone’s argument to make it easier to attack.

There’s a similar straw man towards the end of the letter, where it is said that ‘Dr Hart highlights factors where findings were drawn from the self-reported data, or where the strength of correlation for self-reported data is higher than that found in the annual-returns data - though the direction of correlation is the same - and dismisses them’. It is just not true that I ‘dismiss’ these findings.

The letter opens by saying that ‘Dr Hart's critique was a selective critique of a document designed to provide a non-technical introduction to the research findings for a wide audience.’ I don’t have a problem with this, unless by ‘selective’ they are suggesting that I failed to consider parts of the report which would have changed my conclusions (there are no such parts) or by ‘non-technical’ they are suggesting that, for example, ‘small’ becoming ‘large’ and ‘hypothesis’ becoming ‘finding' are not misrepresentations after all.

More positively, the letter does not try to contradict 7 of my 8 concluding points, nor my overall conclusion that ‘according to the research, the increase in growth to be expected from the use of these factors will be nowhere near sufficient to halt the relentless generational decline, even if the resources could be found to move every lever as far as possible’.

However, they end by trying to defend the claim to have an evidence base for the Reform & Renewal programme by saying that the Church Growth Research Programme is just one part of the evidence. Yet it has repeatedly been cited as the basis, it is claimed as ‘hard information’ compared with the anecdotal, and it is undoubtedly the most comprehensive and detailed research available.

'From Delusion to Reality' - The Bishop of London responds

The Bishop of London has sent a reply to me concerning my paper 'From Delusion to Reality' in which he says:

'It is clearly a substantial piece of work and I shall ask for it to be put on the agenda for the next meeting of the Spending Plans Task Group.'

The Church Growth Research Project reports to the Spending Plans Task Group.

The Spending Plans Task Group reports to both the Archbishops' Council and the Church Commissioners. It was set up 'to help the National Church to reflect on the way that the Church uses its resources'. (See here.)

In March 2014 the membership of the SPTG was as follows (see here, footnote to p.1):
Bishop of London (Chair), Bishop of Sheffield, Dean of York, Christine Hardman, Andreas Whittam Smith, Hywel Rees-Jones, Mike Eastwood, John Spence, Philip James (Secretary).

I don't know the current membership, but at the February 2015 General Synod a question was asked about the membership of Task Groups (see here on Thinking Anglicans) and William Fittall replied saying that he had placed on the noticeboard the membership of various groups, including SPTG. Maybe there is a GS member out there who has kept a record.

See previous posts for more background on my paper.

Church Times coverage of 'From Delusion to Reality'

There's a news report here, which includes my conclusions at the end (p5 in the print edition).

The paper is also the subject of the Leader ('Lost in Translation'), behind the pay wall here.

'Church leaders can be heard regularly saying, "We now know what causes growth." An examination of the evidence by the Revd Dr Mark Hart, a rector in the diocese of Chester, From Delusion to Reality, published this week, suggests otherwise.' 

From Even More Delusion to Reality

The C of E press release issued when 'From Anecdote to Evidence' was published was even more false than the report itself:

"Researchers have concluded that while there is no single recipe, there are common ingredients strongly associated with growth in churches of any size, place or context:
These include: Leadership; A clear mission and purpose; Willingness to self-reflect and learn continually; Willingness to change and adapt according to context; Lay as well as clergy involvement and leadership; Being intentional about prioritising growth; Actively engaging children and teenagers; Actively engaging with those who might not usually go to church; Good welcoming and follow up for visitors; Commitment to nurturing new and existing Christians; Vision" (my bold)

Every one of these factors, it is claimed, according to a professional research conclusion, is strongly associated with growth, in any church in the land.
This is so untrue, as my paper shows (and see previous post).
This is not an academic argument. It is about the people and clergy of the parishes. They are affected by decisions made on the basis of these 'findings'. They deserve better.
 

From Delusion to Reality

The Church of England published the findings of the Church Growth Research Programme (2011-2013) in the report 'From Anecdote to Evidence'.

I have written a paper  'From Delusion to Reality' which evaluates the report.

It looks at how accurately 'From Anecdote to Evidence' reflects the research carried out by David Voas and his team at the University of Essex. Their full report is available here.

The differences are alarming. My paper shows that 'From Anecdote to Evidence' systematically misrepresents or misinterprets the underlying report by David Voas & Laura Watt, thereby exaggerating the usefulness of the findings for numerical growth.

This has implications for the 'Reform & Renewal' programme (involving many Task Groups) which plans to borrow an estimated £100m from the future, on the evidence of this research, to invest in church growth.

Read the full paper here.

(Slightly modified version to the paper - a name correction - at 13.20pm 13/4/15.)