New study questions the validity of publishing hospital mortality rates

September 24, 2007

A previous study of mortality rates for congenital heart surgery used routinely available hospital data that were misleading, according to a report published today on bmj.com which questions the validity of such data being made public.

Professor Westaby and colleagues found the system of information gathering used in the study had underestimated the number of infant deaths. In the previous BMJ study, published in 2004, Oxford had been singled out as having significantly higher mortality than the national average for open heart surgery on infants. Yet this new paper, using data from a different source - the Central Cardiac Audit Database - shows that the hospital’s mortality statistics were not actually different from the mean for all the centres (10 percent compared to 8 percent between 2000 and 2002).

The authors looked at a report from the ‘Dr Foster’ unit at Imperial College which was published in the wake of the inquiry into the Bristol congenital heart surgery deaths. That inquiry, which drew widespread publicity and had a profound effect on surgical practice in the UK, used Hospital Episode Statistics (HES) to compare mortality rates among cardiac surgical units across the country. The 2004 study by Dr Aylin described these mortality statistics.

The authors of the current study compared the mortality rates reported by the administrative HES database and an alternative system, the clinically based Central Cardiac Audit Database, for infants under 12 months undergoing cardiac operations. The statistics were gathered between 1st April 2000 and 31st March 2002.

They found HES did not provide reliable patient numbers or 30-day mortality data. On average HES recorded 20 percent fewer cases than CCAD and only captured between 27 percent and 78 percent of 30-day deaths, with a median shortfall of 40 percent.

In Centre A, with the largest number of operations, 38 percent of all patients were missed by HES and only 27 percent of the total deaths were recorded. Overall, mortality statistics were underestimated by 4 percent using HES data.

The authors say publication of inaccurate statistics detracts from public confidence and that: “If mortality statistics are to be released their quality must be beyond reproach.”

They acknowledge the media are keen to publish such statistics and pinpoint ‘Dr Foster’ who have pioneered this by providing newspapers with information on heart disease, for example, in return for a fee.

They conclude: “Given the problems with data quality, the imprecision of risk stratification models, and the confrontational agenda in the media, we question the value of placing mortality statistics in the public domain.”

Source: BMJ-British Medical Journal