Tuesday, 1 March 2011

Surveys, anonymity and Cath News

Over at Cath News, Michael Mullins' (yet again) distorted report of my piece on the survey of priests released a few days ago seems to have stirred up the usual liberal mafia. 

No doubt Mr Mullins pieces  boost my circulation a little, and maybe some of his readers might be made to think a little if they come here.  But on the whole I've been rather more pleased at (and gained more readers from!) continued mentions on the excellent Pulpit, and recently the Catholic Herald's daily blog round-up!

The liberal fightback

Still, on this occasion I am going to respond to Cath News here at least, because the attempt to clean up Cath News, and put some pressure on for orthodoxy in its selection of articles, is being viciously attacked in the comments box over there, and without any of the censorship Ms Hogan seems to exercise when it comes to conservative commenters.

Take a look for example, at the comments on William Oddie's defense, from the Catholic Herald, of the new missal.  Even having the temerity to post an article defending the hard work done by some of our bishops amongst others, and the decisions of the hierarchy on this subject, incurs the wrath of members of the liberal establishment such as Ms Elizabeth Harrington (I assume of the Brisbane Liturgical Commission).

Anonymity

And Mullins and commenters had another wack at me for my (alleged) anonymity.  So for the record (yet again) let me say this.

I am not 'Incognita' - I previously posted as Terra, although I revealed my actual identity online a couple of years back.  These days I sign my posts as 'Kate' and there has been exactly as much information as cath news requires of its commenters (ie name and location) on the blog for several months now. 

However, just like the vast majority of writers in the mainstream and alternative media, I expect to be judged the content of my posts not my credentials.  If I said, for example, that in fact I have a Masters degree in theology (I do), are those who disagree with me going to be suddenly be persuaded of the validity my views? I don't think so!

That priest survey

And do I really need to set out that I actually do have some academic qualifications in statistics, and experience in designing, running and interpreting surveys in order to point out that a one-third response rate is not necessarily representative?  Surely not!

But I guess ignorance on the norms of social research is to be expected, hence Mullins' and a commenter attack my point that the survey response rate was not high enough to make it representative (though certainly high enough to be worrying).

So for 'PM' and Michael Mullins, a quick lesson in survey design (because yes, I do actually know the basics of this subject!).  There are basically two ways of designing surveys to get representative responses.

1.  Do a census - try and get everyone to respond.
2.  Structure a representative sample - select people by criteria such as in a survey like this, probably age, urban/regional, perhaps seminary attended, and diocese.

In the second case, if people don't respond, you choose someone else from the same demographic until you fill up the 'quota'.  It won't give perfect results, but it will be possible to predict how representative the results are.

In the case of a census as this one appeared to be, you have a problem with sample selection bias - those who choose to fill out the form may have very different characteristics from the norm.  So you would normally need a response rate of more like 75-80% rather than a third they actually got to be confident you had a reasonably representative group.  Now of course there are ways of adjusting for these problems - so to be definitive, one would need to know more about how the authors actually structured their approach to priests and analysed the data.  Still, based on what was actually said in the media report, I stand by my comment!

So what do we do about Cath News?

I'm afraid I can't personally bring myself to bother subscribing (except via twitter, which apparently isn't enough to gain an invitation to participate in their subscriber surveys) or commenting in the cath news comments boxes.  Until there is a management change, I think it is pretty much a lost cause and a waste of time better spent elsewhere.  Still, as it does reach so many people, and because it is a semi-official publication, we should keep the pressure for reform on...so if you do subscribe/comment over there, keep trying!

2 comments:

PM said...

I'm not the offending commentator from Eden,by the way.

Any survey run by Chris McGillion would be pretty likely to be self-selecting. This is the theological genius who thinks we 'only' need to get rid of the pernicous influence of St Paul - presumably in oder to rediscover Jesus the dim, distant mythological precursor of Carl Rogers.

The Cathnews discussio on the new translation is becoming a hoot. 'If it is a theology thing then why not save it for the theologians and give the common folk the language of the common person? Vatican II recommended this ... but now a bunch of foreigners (encouraged by a local ambitious cleric or two) who do not even speak the language have ambushed the process,' says one indignant commentator. The Pauline Hanson school of liturgy?

Anonymous said...

It's a waste of time commenting on CathNews, for the reason you mention. But I agree - the pressure must be kept on them because it reaches so many people.