An Intervention to Improve Cause-of-Death Reporting in New York City Hospitals, 2009–2010

Introduction Poor-quality cause-of-death reporting reduces reliability of mortality statistics used to direct public health efforts. Overreporting of heart disease has been documented in New York City (NYC) and nationwide. Our objective was to evaluate the immediate and longer-term effects of a cause-of-death (COD) educational program that NYC’s health department conducted at 8 hospitals on heart disease reporting and on average conditions per certificate, which are indicators of the quality of COD reporting. Methods From June 2009 through January 2010, we intervened at 8 hospitals that overreported heart disease deaths in 2008. We shared hospital-specific data on COD reporting, held conference calls with key hospital staff, and conducted in-service training. For deaths reported from January 2009 through June 2011, we compared the proportion of heart disease deaths and average number of conditions per death certificate before and after the intervention at both intervention and nonintervention hospitals. Results At intervention hospitals, the proportion of death certificates that reported heart disease as the cause of death decreased from 68.8% preintervention to 32.4% postintervention (P < .001). Individual hospital proportions ranged from 58.9% to 79.5% preintervention and 25.9% to 45.0% postintervention. At intervention hospitals the average number of conditions per death certificate increased from 2.4 conditions preintervention to 3.4 conditions postintervention (P < .001) and remained at 3.4 conditions a year later. At nonintervention hospitals, these measures remained relatively consistent across the intervention and postintervention period. Conclusion This NYC health department’s hospital-level intervention led to durable changes in COD reporting.


Introduction
Inaccurate reporting of cause of death (COD) on death certificates limits the validity and usefulness of mortality indicators for policy, research, and applied public health decisions (1,2). Validation studies and audits have found that heart disease is overreported as a COD (3)(4)(5)(6)(7). A comparison of certificates of in-hospital deaths with medical charts for deaths in 2003 in New York City (NYC) showed 91% overreporting of heart disease. Overreporting increased with age by 51% for decedents aged 35 to 74 years, 94% for those aged 75 to 84 years, and 137% for those 85 years or older (7). A previous study of 4 other regions found that 20% of certificates of in-hospital death incorrectly documented heart disease as the underlying COD (6). Because NYC's heart disease risk factors are not greater than those of the rest of nation (8), overreporting likely partially explains NYC's high heart disease death rates (9,10).
Physicians typically report COD for deaths from natural causes. Overreporting may result from lack of training, hospital leadership's failure to emphasize the importance of correctly reporting COD, and survivors' desire to avoid certain diagnoses (11-15). Physicians are not usually trained in COD reporting despite existing training materials and recommendations that such training be conducted (16)(17)(18)(19). Autopsy, physician review panels, and querying (ie, contacting certifiers for clarification) may improve reporting but may not be feasible or cost-effective and therefore may not be adopted (11,20,21). Previous interventions consisting of workshops and interactive training conducted by clinical colleagues have demonstrated short-term improved COD reporting accuracy among trainees (13,14,22-26). Previous reports have not assessed long-term changes in COD reporting or the effect of such changes on population mortality statistics. Our objective was to evaluate immediate and longer-term effects of an educational program in COD documentation that the NYC Department of Health and Mental Hygiene (DOHMH) conducted at 8 NYC hospitals; the evaluation focused on heart disease reporting and the average conditions per death certificate, both of which are indicators of quality of COD reporting.

Methods
We used a time series design to evaluate an intervention to educate physicians on COD reporting that we conducted from June 2009 to January 2010 at 8 NYC hospitals. The intervention and analysis did not pose any risk to living subjects and was conducted as a quality improvement activity; therefore, it did not require institutional review board approval under the NYC Health Code.

Identification of intervention hospitals
In early 2009, DOHMH ranked the 64 NYC hospitals reporting more than 50 deaths per year by their 2008 proportion of heart disease deaths; that is, the ratio of heart disease deaths to total deaths that the hospital reported. We selected 8 hospitals that had the greatest potential to improve citywide vital statistics: 7 hospitals with the highest proportions of heart disease deaths and the hospital with the tenth highest proportion because it was the third largest hospital in NYC and had nearly 1,000 deaths per year. All 8 hospitals agreed to participate. The proportion of death certificates reporting heart disease as COD at these 8 hospitals ranged from 60.4% to 78.2% in 2008, while the average at nonintervention hospitals was 25.2% (Table 1). We compared demographics at intervention and nonintervention hospitals in 2008 using z scores.

Underlying COD
The underlying COD, as the World Health Organization defines it, is "the disease or injury that initiated the chain of events leading directly to the death, or the circumstances of the accident or violence which produced the fatal injury" (27). An International Classification of Diseases (ICD)-10 code is assigned as the underlying COD for each death certificate. ICD-10 codes are determined by applying the standardized 1992 World Health Organization International Classification of Diseases, 10th Revision (ICD-10) algorithm to the cause-of-death text provided by certifying physicians in COD fields (27). The algorithm is applied automatically by the National Center for Health Statistics' Mortality Medical Data System (MMDS) software (National Center for Health Statistics, Hyattsville, Maryland) or manually by a trained nosologist if automated coding fails or is not available (28). We defined heart disease deaths as those assigned ICD-10 codes I00-I09, I11, I13, or I20-I51 (29), which is consistent with National Center for Health Statistics' criteria. These include heart failure, cardiomyopathy, arrhythmias, and acute and chronic rheumatic, hypertensive, ischemic, and pulmonary heart disease.

Intervention
The intervention consisted of 2 main components: a conference call with senior hospital staff, which included medical directors and medical, quality assurance, admitting, and regulatory affairs staff, and an on-site, in-service training of hospital and clerical staff involved in death certification. Other activities included process mapping of death certification and registration workflow, auditing medical records, and promoting an online learning module. A conference call and in-service training were held for each hospital. The first conference call at any hospital was held on June 26, 2009; the last in-service training was conducted January 13, 2010.
During each conference call, DOHMH described unexpected differences between frequencies of key causes of death (eg, heart disease, Alzheimer disease) reported at the hospital compared with frequencies NYC-wide and nationally. DOHMH stressed the importance of accurate COD reporting for policy and research and the legal requirement for data accuracy in NYC's Health Code. DOHMH outlined steps required to address the problem. Hospital personnel agreed to complete the remaining intervention activities.
For process mapping, each hospital documented their death registration workflow following the conference call. DOHMH then proposed hospital-specific action plans, including the identification of staff required to attend in-service training and the revision of hospital policy and protocols.
To highlight deficiencies in COD documentation at each hospital, DOHMH identified a sample of 30 death certificates that the hospital registered preintervention. The sample consisted of 10 randomly sampled certificates in each of 3 categories: certificates reporting only a single heart disease condition as the COD, certificates reporting a heart disease condition and other comorbidities as COD, and certificates that did not include heart disease as a cause of death. We asked hospital staff trained in COD documentation to review corresponding medical charts and report all conditions indicated in the medical record as contributing to the death. We compared medical record information with the death certificate and cited the discrepancies we discovered in the in-service training. We also used audit information to inform the average-number-of-conditions outcome measure as described below.
DOHMH requested that physicians and staff involved in death certification at intervention hospitals complete the Improving Cause of Death Reporting online training module created by DOHMH (30).
DOHMH quality improvement and medical personnel conducted an in-service training with physicians and staff involved in death registration at each intervention hospital. The 45-minute presentation, typically attended by 30 to 100 residents and staff, addressed legal requirements for death registration; compared COD distributions at the hospital, throughout NYC, and nationwide; gave examples of discrepancies between death certificate cause of death and the medical record identified during each hospital's audit; and provided generic examples of proper and improper COD documentation, with emphasis on heart disease. A question-and-answer session followed, and we distributed DOHMH'S City Health Information (31) bulletin on improving cause of death reporting.
We defined 2 primary outcome measures: 1) the proportion of heart disease deaths reported on death certificates, which is an indicator of the intervention's effect on heart disease overreporting, and 2) the average number of conditions reported on death certificates, defined as the number of entity axis codes documented in Parts I and II of the certificate. Entity axis codes are ICD-10 codes assigned to the conditions the physician wrote on the death certificate, processed according to the MMDS algorithm. The algorithm processes and codes the conditions reported by the physician in the cause of death section while eliminating redundancies and inconsistencies (32). On the basis of the DOHMH audit, the number of entity axis codes is an indicator of sufficient detail and COD reporting quality. We classified audited death certificates as inaccurate if the underlying COD on the death certificate was not reported in the medical record or as accurate if the underlying COD on the death certificate was reported in the medical record. Inaccurate death certificates on average reported fewer conditions compared with accurate certificates (1.45 vs 1.75, P = .07, n = 147, t test 2-sample equal variance).
For statistical analysis we defined the preintervention period as January 1, 2009, through June 30, 2009; the active intervention period as June 30, 2009, through December 31, 2009; the postintervention period as January 1, 2010, through June 30, 2010; and the extended postintervention period as January 1, 2011, through June 30, 2011. We used January-through-June deaths each year to control for seasonal variation in COD. The primary analysis evaluated change in the 2 outcome measures at intervention hospitals between the pre-and postintervention periods and the preintervention and extended postintervention periods by using a z score for the difference between 2 proportions with pooled variance estimates. Secondary analyses repeated these outcome measure comparisons within nonintervention hospitals and citywide. To understand any trends unrelated to the intervention, we calculated the outcome measures for each calendar year between 2001 and 2010 at intervention hospitals individually and combined, at nonintervention hospitals, and citywide. We used SAS version 9.2 (SAS Institute, Inc, Cary, North Carolina) for all analyses.

Results
In 2008, the 8 intervention hospitals reported 28% of heart disease deaths and 13% of all-cause deaths among all NYC hospitals with more than 50 deaths per year. At intervention hospitals, the number of deaths ranged from 297 to 1,197 for a total of 4,597; the proportion of heart disease deaths ranged from 60.4% to 78.1%, averaging 68.2%; and the average number of conditions reported per death certificate ranged from 1.7 to 2.7, averaging 2.4 (Table 1). Nonintervention hospitals reported 30,736 deaths in 2008 with 25.2% due to heart disease and reported an average of 3.0 conditions per certificate. Decedents at nonintervention hospitals were, on average, younger and less likely to be non-Hispanic white than at intervention hospitals.
At intervention hospitals the proportion of heart disease deaths decreased from 68.8% preintervention to 32.4% postintervention (P < .01). At each hospital, the proportion ranged from 58.6% to 79.5% in the preintervention period and from 25.9% to 45.0% in the postintervention period. The decrease was significant (P < .01) at each hospital; the absolute difference ranged from 27.8 to 52.4 percentage points ( Table 2). The average proportion of heart disease deaths in the extended postintervention period ranged from 23.5% to 50.0%, significantly lower than preintervention at each hospital. In comparison, the proportion of heart disease deaths at nonintervention hospitals remained relatively consistent: 26.6% preintervention, 24.4% postintervention, and 22.1% extended postintervention. Citywide, the proportion of heart disease deaths decreased from 39.1% preintervention to 34.2% postintervention and to 32.5% for the extended postintervention period.
At intervention hospitals combined, the average number of conditions reported increased from 2.4 preintervention to 3.4 postintervention ( Table 3). The absolute increase at each hospital in the preintervention period ranged from 0.75 to 1.33 conditions on average. The average number of conditions reported in the extended postintervention period remained higher at each intervention hospital, an average of 3.4 across hospitals. In comparison, nonintervention hospitals reported a relatively consistent average number of conditions: 2.9 preintervention, 2.9 postintervention, and 3.0 extended postintervention. Citywide, the average number of conditions reported per death certificate increased from 2.7 preintervention to 2.8 postintervention and 2.9 extended postintervention.
The proportions of deaths from heart disease had been stable for several years before this intervention at both intervention hospitals and nonintervention hospitals (Figure 1). Similarly, the magnitude of change in the average number of conditions reported was unprecedented at any of the hospitals before the intervention and similar among intervention hospitals following the intervention (Figure 2).

Discussion
This hospital-level intervention is the first to demonstrate immediate and durable changes in COD reporting with a reduction in heart disease overreporting and an increase in the average number of conditions reported. The reporting changes at the 8 intervention hospitals were so pronounced that citywide outcome measures were notably different before and after the intervention.
Fourteen previous reports have measured effectiveness of COD educational efforts (33); only 1 study was conducted in the United States (21). Some studies asked trainees to complete death certificates for fabricated cases immediately preand postintervention, an approach that limits generalizability to other causes of death and does not evaluate sustained intervention effects (33). Other studies compared COD documented on the death certificate with the medical record before and after intervention, a resource-intensive approach. In those reports, hospital-affiliated physicians provided training (33). We add to the literature by showing that public health staff vested in vital statistics data can also effect change and that interactive training can have a sustained effect on COD reporting.
Our study has some limitations. The average number of conditions reported was correlated with data accuracy preintervention, but this may not be generalizable to other settings. Our intervention hospitals had significant heart disease overreporting; thus, our findings may not be generalizable to hospitals with a lesser degree of overreporting or different types of quality issues. Another limitation is that we did not query physicians postintervention to learn whether the in-service training met their COD reporting needs.
Given the volume of deaths occurring at intervention hospitals (375 per month on average), resources did not permit direct comparison of death certificates with the medical record except in our preintervention sample. The inability to definitively conclude postintervention accuracy is a study limitation. We did establish that hospitals overreporting heart disease preintervention reported fewer conditions on average and that inaccurate death certificates of overreporting hospitals reported fewer conditions than their accurate death certificates reported. Thus, the observed increase in average number of conditions reported at intervention hospitals suggests improved COD reporting. Additionally, the number of deaths from other leading causes postintervention increased proportionately as expected, suggesting improved reporting rather than a shift from heart disease to a single or few random erroneous reported causes of death (34). An alternate explanation is that deaths inaccurately reported preintervention as heart disease continued to be reported inaccurately, but in proportion to NYC's leading causes of death, which seems less likely.
Our analyses of 10-year trends in these outcome measures and our comparison of outcome measures pre-and postintervention at nonintervention hospitals establishes that the decrease in heart disease deaths and the increase in the average number of conditions postintervention is restricted to intervention hospitals and is not likely due to a secular or historical trend. Although the decrease in heart disease proportions was statistically significant in the nonintervention group, because approximately 14,000 deaths occurred during each observation period, the decrease is an order of magnitude less than at intervention hospitals. The slight decrease in heart disease deaths and increase in average number of conditions reported at nonintervention hospitals between postintervention and extended postintervention periods may reflect DOHMH's continued efforts to improve COD reporting citywide.
Although bias or confounding might explain results in any nonrandomized study, neither can fully explain our results, unless the true heart disease death rates decreased to this degree in populations served by intervention hospitals only during our study period, and we coincidentally embarked on a campaign to improve COD reporting during this period. Another possible but unlikely explanation is regression to the mean; that is, because we selected hospitals based on their high percentages of deaths from heart disease, by chance this percentage was closer to the average upon second measurement. Previous years' data do not support this explanation because intervention hospitals historically had reported high proportions of heart disease and nonintervention hospitals had reported low proportions.
Decedents at intervention hospitals were older and more likely to be non-Hispanic white than at nonintervention hospitals. While heart disease risk also varies by these factors and COD reporting quality varies by age, this variation cannot explain the intervention's positive results. We compared the recent outcome measures over time within each intervention hospital so that differences in decedents' characteristics by hospital are not confounding our primary comparison. Furthermore, an analysis of changes in all reported causes of death following the intervention and controlling for decedents' characteristics did not alter the intervention's observed effect (34).
This intervention incorporated hospital-specific policy, practices, and educational components to achieve the support of staff and administration and was completed at little cost. The primary expenditure, outside of the staff time of DOHMH personnel, consisted of minimal travel expenses. One full-time epidemiologist devoted approximately 50% of her time to developing content over the course of 18 months with input from subject matter experts. The director of the Office of Vital Statistics conducted the conference calls. The director and a DOHMH physician conducted the in-service trainings. Health departments with fewer resources or those that cover a larger geographic area may be able to improve COD reporting quality without extensive travel by using a conference call alone.
One key benefit of on-site, in-service training was qualitative feedback from a larger audience. Most doctors reported no prior training in death certification. Additionally, physicians and staff expressed frustration over the past DOHMH practice of rejecting certificates based on COD and suggested that funeral directors, affected by death registration delays, may proactively request certain "safe" causes of death, such as atherosclerotic heart disease, to avoid DOHMH rejections. Physicians also perceived validation checks in the Electronic Death Registration System (EDRS) as obstacles to death registration. DOHMH has reduced these barriers citywide. As of March 2010, the death registration protocol requires rejection only if the cause of death does not appear natural or the only COD reported is a mechanism (eg, cardiopulmonary arrest, asystole, respiratory arrest). As of October 2009, physicians can override many COD related EDRS validation checks. As of January 2010, the NYC Health Code requires all EDRS users to complete an online training on COD documentation, which may explain some postintervention changes in outcome measures among nonintervention and intervention hospitals. However, compliance with this training requirement has been poor, and further efforts are planned to improve enforcement. DOHMH has also discussed the importance of accurate cause of death reporting and physician autonomy at meetings of NYC funeral director associations. These and other citywide efforts may explain some COD reporting changes at intervention hospitals and at nonintervention hospitals.
In NYC, medical residents complete many death certificates. Sustained improvement in COD reporting will depend on hospital and residency administrations' support of continuous quality improvement. At some hospitals, improvements waned in the expanded postintervention period, indicating that ongoing training is needed to ensure that new staff and medical residents understand COD documentation. On the basis of the success of this intervention, DOHMH conducted conference calls and in-service trainings in 12 additional hospitals in 2011. As resources permit, DOHMH will reach all NYC hospitals. Other completed COD improvement initiatives include issuance of COD data quality reports; dissemination of education materials such as physician pocket cards and COD posters; assisting via telephone during weekdays; and monitoring DOHMH death registration rejections.
National efforts to improve COD reporting quality are ongoing. In partnership with National Association for Public Health Statistics and Information Systems (NAPHSIS) and the National Center for Health Statistics, DOHMH developed an e-learning course on COD completion for national use, which is available to NAPHSIS members. Other jurisdictions can customize the national module.
Inaccurate COD reporting occurs at local, state, and national levels. Many researchers use mortality data; therefore, poor quality COD reporting, including heart disease overreporting, affects the usefulness of public health policies, spending, and programs informed by the data. We have demonstrated that a health department can reduce heart disease overreporting with a training intervention. DOHMH continues to expand COD training in NYC. Other US jurisdictions should consider similar interventions to address this critical national problem.  P values calculated by using a 2-tailed test. All tests of difference use a standard normal approximation of the sampling distribution of the estimates with their associated z scores to test the null hypothesis of equal means.

Post-Test Information
To obtain credit, you should first read the journal article. After reading the article, you should be able to answer the following, related, multiple-choice questions. To complete the questions (with a minimum 70% passing score) and earn continuing medical education (CME) credit, please go to http://www.medscape.org/journal/pcd . Credit cannot be obtained for tests completed on paper, although you may use the worksheet below to keep a record of your answers. You must be a registered user on Medscape.org. If you are not registered on Medscape.org, please click on the "Register" link on the right hand side of the website to register. Only one answer is correct for each question. Once you successfully answer all post-test questions you will be able to view and/or print your certificate. For questions regarding the content of this activity, contact the accredited provider, CME@medscape.net. For technical assistance, contact CME@webmd.net. American Medical Association's Physician's Recognition Award (AMA PRA) credits are accepted in the US as evidence of participation in CME activities. For further information on this award, please refer to http://www.ama-assn.org/ama/pub/category/2922.html . The AMA has determined that physicians not licensed in the US who participate in this CME activity are eligible for AMA PRA Category 1 Credits™. Through agreements that the AMA has made with agencies in some countries, AMA PRA credit may be acceptable as evidence of participation in CME activities. If you are not licensed in the US, please complete the questions online, print the AMA PRA CME credit certificate and present it to your national medical association for review.