The longitudinal study of turnover and the cost of turnover in emergency medical services.

Abstract Background. Few studies have examined employee turnover and associated costs in emergency medical services (EMS). Objective. To quantify the mean annual rate of turnover, total median cost of turnover, and median cost per termination in a diverse sample of EMS agencies. Methods. A convenience sample of 40 EMS agencies was followed over a six-month period. Internet, telephone, and on-site data-collection methods were used to document terminations, new hires, open positions, and costs associated with turnover. The cost associated with turnover was calculated based on a modified version of the Nursing Turnover Cost Calculation Methodology (NTCCM). The NTCCM identified direct and indirect costs through a series of questions that agency administrators answered monthly during the study period. A previously tested measure of turnover to calculate the mean annual rate of turnover was used. All calculations were weighted by the size of the EMS agency roster. The mean annual rate of turnover, total median cost of turnover, and median cost per termination were determined for three categories of agency staff mix: all-paid staff, mix of paid and volunteer (mixed) staff, and all-volunteer staff. Results. The overall weighted mean annual rate of turnover was 10.7%%. This rate varied slightly across agency staffing mix (all-paid == 10.2%%, mixed == 12.3%%, all-volunteer == 12.4%%). Among agencies that experienced turnover (n == 25), the weighted median cost of turnover was $$71,613.75, which varied across agency staffing mix (all-paid == $$86,452.05, mixed == $$9,766.65, and all-volunteer == $$0). The weighted median cost per termination was $$6,871.51 and varied across agency staffing mix (all-paid == $$7,161.38, mixed == $$1,409.64, and all-volunteer == $$0). Conclusions. Annual rates of turnover and costs associated with turnover vary widely across types of EMS agencies. The study's mean annual rate of turnover was lower than expected based on information appearing in the news media and EMS trade magazines. Findings provide estimates of two key workforce measures—turnover rates and costs—where previously none have existed. Local EMS directors and policymakers at all levels of government may find the results and study methodology useful toward designing and evaluating programs targeting the EMS workforce.

direct and indirect costs through a series of questions that agency administrators answered monthly during the study period. A previously tested measure of turnover to calculate the mean annual rate of turnover was used. All calculations were weighted by the size of the EMS agency roster. The mean annual rate of turnover, total median cost of turnover, and median cost per termination were determined for three categories of agency staff mix: all-paid staff, mix of paid and volunteer (mixed) staff, and allvolunteer staff. Results. The overall weighted mean annual rate of turnover was 10.7%. This rate varied slightly across agency staffing mix (all-paid = 10.2%, mixed = 12.3%, all-volunteer = 12.4%). Among agencies that experienced turnover (n = 25), the weighted median cost of turnover was $71,613.75, which varied across agency staffing mix (allpaid = $86,452.05, mixed = $9,766.65, and all-volunteer = $0). The weighted median cost per termination was $6,871.51 and varied across agency staffing mix (all-paid = $7,161.38, mixed = $1,409.64, and all-volunteer = $0). Conclusions. Annual rates of turnover and costs associated with turnover vary widely across types of EMS agencies. The study's mean annual rate of turnover was lower than expected based on information appearing in the news media and EMS trade magazines. Findings provide estimates of two key workforce measures-turnover rates and costs-where previously none have existed. Local EMS directors and policymakers at all levels of government may find the results and study methodology useful toward designing and evaluating programs targeting the EMS workforce. Key words: emergency medical services; turnover; cost of turnover; longitudinal study

INTRODUCTION
Recruitment and retention are two of the most salient issues in emergency medical services (EMS) today. Reports of inadequate staffing in EMS agencies appear frequently in the local news media. [1][2][3][4][5][6][7] The National Association of State EMS Directors, a group formed in 1980 to represent state and regional EMS leaders, has twice identified recruitment and retention as the most important issues facing rural EMS agencies across the nation. 8,9 Prominent organizations such as the National Council of State Legislators and the National Rural Health Association have also expressed concern for the viability of our nation's EMS workforce. [10][11][12] Despite widespread concern and federal initiatives (e.g., the Workforce Project led by the National Highway Traffic Safety Administration and EMS Grant

209
Component of the Critical Access Hospital Program), we have an imperfect understanding of these problems on a national scale, as research on EMS recruitment and retention is limited.
In fact, we identified only one peer-reviewed study that characterized the magnitude of EMS recruitment and retention issue nationally from the local EMS director perspective. In this cross-sectional survey of local EMS agency directors, Freeman and colleagues determined that approximately half of all agencies surveyed were not fully staffed; 37% of directors said recruitment was always a problem, and 55% reported difficulty with retention. 6 This study showed that EMS director-reported problems with recruitment and retention were more common in rural versus urban areas. In a 2007 non-peer-reviewed study supported by JEMS, it was reported that approximately 15% of fulltime, 23% of part-time, and 18% of volunteer EMS workers leave their jobs annually. 13 This study was cross-sectional, with an overall response rate of 13%. Our understanding of turnover in EMS is limited to a narrow body of research and information.
The primary objective of the study presented here was to provide more comprehensive information on the EMS workforce by quantifying the annual rate of turnover in EMS using a longitudinal study design and nationwide sample of EMS agencies. We also quantify the costs associated with EMS turnover using a modified version of the Nursing Turnover Cost Calculation Methodology (NTCCM). 14,15

METHODS
We used a mixed-methods approach and drew from multiple sources of EMS agency and individual data to develop an in-depth descriptive picture of turnover and associated costs in EMS. We used a stratified random-sampling procedure to recruit a diverse sample of EMS agencies. Next, we used a longitudinal observational study design to capture information related to turnover and costs associated with turnover. Finally, we used the comparative method to present our findings. 16 The comparative method is widely used in the analysis of policy-relevant issues, and is the preferred method for studies comparing a small yet diverse sample of observations using a limited set of statistics (e.g., descriptive statistics). 16 We compared findings across categories of EMS agency staffing mix (i.e., all-paid staff, a mix of paid and volunteer staff, or all-volunteer staff). This study was approved by the University of Pittsburgh Institutional Review Board.

Study Sample
A list of EMS agencies was created from two sources: 1) the Commonwealth of Pennsylvania Office of EMS list of licensed EMS agencies and 2) a nonprofit EMS purchasing cooperative with 1,000 member agencies in 26 states. This list of approximately 2,000 EMS agencies was stratified by rural and urban status based on the Rural-Urban Commuting Area (RUCA) designation. 17 We assigned each agency a unique ID number using a random-number generator in SAS version 9.1 (SAS Institute, Inc., Cary, NC). The list was sorted by this randomly generated number from lowest to highest. From this list, we selected the first 400 rural and first 300 urban agencies to receive study recruitment materials. We anticipated that between 10% and 50% of agencies would respond to a mass-mailing invitation and be screened for eligibility. 6,18,19 We oversampled rural agencies based on an expectation that many were nontransporting agencies and thus ineligible for participation. Because our study was descriptive and limited by available resources, we enrolled a maximum of 50 EMS agencies. A power calculation was not performed because differences in outcome measures were not hypothesized across variables of interest.
All study recruitment materials were mailed during the first week of January 2008 and contained two letters. The first letter was from the principal investigator (PI) and explained the study. The second letter was from a nationally recognized EMS figure who expressed support for our study.
An EMS agency was eligible for participation if it satisfied the following criteria: 1) was a provider of ground ambulance services and had access to all agency expense and revenue data linked to the provision of ground transportation and operations and was able to distinguish these costs from other services provided (agencies with fire or multiconsortium company affiliations were thought less likely to meet this criterion and therefore were carefully screened prior to enrollment); 2) was willing to assign an agency contact to be in charge of this study and allow this individual two to three hours per month for a total of six months to collect and enter data via a secure website; 3) had access to the Internet; and 4) engaged in patient transportation (emergent, nonemergent, or both).

Study Protocol
The enrolled agencies were given a user name and password to a secure website where agency data were collected. Upon their first time logging in, agency contacts were presented with a series of financial and human resource questions. These questions were asked only once and addressed the following: 1) total agency revenue and expenses for 2007; 2) the amounts in dollars charged for ambulance transportation and average reimbursement for each of the six transportation categories (e. . Agency contacts used the study's secure website to populate a virtual employee roster. At the employee level, we collected the following pieces of information for each agency employee: 1) hourly pay or annual salary; 2) certification level (e.g., first responder, EMT-B, paramedic, firefighter only); 3) average hours worked per week; 4) full-time status; and, 5) role as an administrator.
Agency contacts were instructed to access the website on the last week of each month to update agency rosters, document terminations, open positions, and new hires, and answer questions related to the cost of turnover. These questions were developed by our study team and informed by the NTCCM, a methodology developed by a member of our study team (CBJ). 14,15,[20][21][22] The NTCCM was derived from methodologies used in the business and nursing professions, and is grounded in human resource accounting methods. [23][24][25] It was first applied in 1988. 14,15,[20][21][22] The NTCCM combines cost calculation methodologies developed by Hall, Hoffman, and Flamholtz to estimate prehire and posthire costs associated with nursing turnover. 20,23,24,25 Prehire costs include expenses associated with advertising and recruitment, costs associated with vacancies, and hiring costs. Posthire costs include expenses associated with orientation and training of new hires, costs associated with new-hire productivity, pretermination costs associated with terminated employees, and costs associated with the termination process. Prehire and posthire costs include both direct costs (e.g., newspaper advertisements) and indirect costs (e.g., time spent interviewing candidates). The NTCCM has undergone rigorous content, construct, and face validity testing. 21 Both content validity and construct validity refer to the content/substance and structure of NTCCM questions. Items that have sound content and construct validity coalesce on defined constructs and measure what the items were intended to measure. Face validity refers to the interpretability of NTCCM items by key informants (e.g., nurse leaders who maintain data sources). All forms of validity were evaluated by an expert panel that included economists knowledgeable of human capital, finance experts knowledgeable of resource accounting methods, human resource managers knowledgeable of staffing and turnover, and nurse executives.
We used the NTCCM as a framework to develop our cost-of-turnover questions. All questions were reviewed by our team, which included health services researchers, a labor economist, EMS researchers, and experts in nursing turnover. This exercise addressed content validity and construct validity. We evaluated face validity by pilot testing our questions in three EMS agencies located in the surrounding Pittsburgh, Pennsylvania, area. Agency contacts were asked to review item grammar, item interpretation, and relevance to diverse EMS operations. Following pilot testing, the University of Pittsburgh Center for Research on Healthcare Data Center created an electronic and secure web-based version of all questions.
Data collection began on the last week of January 2008. We instructed agency contacts to access the study website and answer study questions at the end of each month. We tracked compliance with instructions using an automated e-mail notification mechanism. Whenever an agency completed a monthly data entry session, an e-mail was sent to the PI and study staff. We provided all agencies with technical assistance and on-site visits when requested. Notification of monthly data entry completion was followed with the release of an agency incentive. In total, each agency received $250 over the six-month study period.

Variable Descriptions and Analysis of Data
Agency demographic variables were documented for descriptive purposes. We defined an agency as rural if more than 50% of its service area ZIP codes were linked to categories 4.0-10.6 of the RUCA coding scheme or if the county in which the agency was located was designated as a micropolitan Core-Based Statistical Area (CBSA) or non-CBSA by the Office of Management and Budget (OMB). 26 We used these data to group agencies into four major U.S. Census regions (Midwest, Northeast, South, and West). Agencies were placed into one of four categories of workforce size (<25 employees and volunteers, 25-49, 50-100, and >100). We defined the staffing mix of agency employees as all-paid staff (agencies with all employed persons earning a wage), a mix of paid and volunteer staff (agencies with a mix of wage earners and non-wage earners), or allvolunteer (agencies with paramedics and EMTs receiving no specified wage). In cases where only the agency directors were documented as earning a wage, the agency's status was classified as all-volunteer. Agencies self-selected their model type as hospital-based, government/third-service, private freestanding, or fire-based EMS. The total number of agency dispatches in 2007 were grouped into the following categories : 1-500 responses, 501-1,000, 1,001-5,000, and >5,000. Agency expenses and revenue in 2007 were classified as follows: <$500,000, $500,000-$1,000,000, and >$1,000,000. The proportion of total agency expenses linked to employee salaries and benefits was stratified into two categories: <60% and ≥60%. Total agency revenue linked to Centers for Medicare & Medicaid Services (CMS) was classified as follows: no CMS revenue, <50% CMS revenue, and ≥50% CMS revenue.
Our three main outcomes of interest included the annual rate of turnover, the projected annual costs of turnover, and the cost per termination. We calculated the annual rate of turnover as (the total number of terminations during the six-month study period × 2)/ total number of employees. 27 To calculate the cost of turnover, we first excluded all agencies that did not experience turnover during the study period. We then calculated the monthly prehire and posthire costs and adjusted these figures by the number of open positions, new hires, and terminations ( Table 1). The prehire and posthire costs were summed to identify the total cost of turnover over the study period. This six-month figure was then multiplied by 2 to represent the total projected annual cost of turnover. We calculated the average cost per termination by dividing the total cost of turnover by total terminations.
We analyzed study data using descriptive statistics (e.g., frequencies, means, medians, and lower 25% quartile [Q1] and upper 75% quartile [Q3]). We calculated the average rate of turnover across agencies, the median total projected cost of turnover, and the median cost per termination. We weighted each measure by the total number of positions at each agency to give more weight to those agencies with greater numbers of employed persons. Results are presented across a three-level agency demographic variable, staffing mix (all-paid staff, mix of paid and volunteer staff, and all-volunteer staff). All calculations and descriptive statistics were performed in SAS version 9.1.

Sample Demographics
We received 70 responses (10%) from agencies interested in participating in our study. Sixteen agencies did not meet the eligibility criteria and seven failed to complete the enrollment process (e.g., expressed interest but were unreachable immediately afterwards). Of the 47 agencies enrolled in the study, six failed to complete the required six months of data entry. Among these agencies, lack of time was the most commonly cited reason for attrition. At one agency, the contact person left the agency and the agency leadership indicated that no suitable replacement was available. At a separate agency, the contact person cited health reasons, the need to take extended leave from work, and lack of a suitable replacement to assign to our study. Results from a nonrespondent analysis showed no significant differences in rural status or county population across three categories of response and enrollment: 1) enrolled and completed the study; 2) enrolled but became lost to attrition or ineligible upon screening; and 3) no response to the recruitment packet.
We excluded one additional agency from all calculations because this agency experienced an uncharacteristic pattern of patient transports during the study period (n = 0 patient transports). Calculations were performed on 40 EMS agencies.
Approximately 25 agencies (62%) employed an allpaid staff, 25% employed a mix of paid and volunteer staff, and 13% employed an all-volunteer staff ( Table  2). Among agencies with an all-paid staff model, most were classified as rural and were located in the Midwest Census region. A large proportion of these agencies employed less than 25 staff, were self-described as being hospital-based, dispatched between 1,001 and 5,000 ambulances in 2007, and had an annual budget (revenues and expenses) of $1 million annually. Fiftypercent of agencies with a mix of paid and volunteer staff were located in the Northeast Census region. Most of these agencies employed between 25 and 49 employees, were described as private freestanding, dispatched between 501 and 1,000 ambulances in 2007, and had an annual budget less than $1 million annually. Of the remaining all-volunteer EMS agencies, most were located in the Midwest Census region, all employed less than 25 total staff, most were described as government or third-service that dispatched less than 500 ambulances in 2007, and most had an annual budget that totaled less than $500,000 annually.

Annual Rate of Turnover
There were 106 total terminations across all agencies studied (n = 80 for all-paid agencies, n = 20 among mixed staffing agencies, and n = 6 among all-volunteer models). Approximately 25 agencies (62.5%) experienced turnover during the six-month study period ( Table 3). The average number of terminations for allpaid EMS agencies (n = 25) was 3.2. The average number of terminations for mixed staffing agencies (n=10) was 2.0, and that among all-volunteer agencies (n = 5) was 1.2. The overall weighted average annual rate of turnover was 10.7% (standard deviation [SD] 10.3). This rate was lowest among agencies using an all-paid staff (10.2%), and was followed by agencies with a mix of paid and volunteer staff (12.3%) and agencies using all-volunteer staffing (12.4%).

Cost of Turnover
The projected median annual total cost of turnover across all agencies that experienced turnover was $71,613.75 (Table 4). Total median costs varied across agency staffing mix, with agencies using an allpaid staffing having the highest median total cost at $86,452.05 and all-volunteer staffing the lowest at $0. Among the 25 EMS agencies that experienced turnover, vacancy costs (e.g., labor and expenses linked to overtime due to understaffed conditions) and new-hire productivity costs (e.g., time required of new hires to reach cut-loose status) contributed the most to overall cost. Among EMS agencies with a mix of paid and volunteer staff, vacancy costs and costs associated with orientation and training were the two largest

Prehire costs
Advertising and recruitment costs The sum of 1) labor and travel and other expenses associated with job fairs, student visitation, community recruitment; 2) newspaper and Internet advertisements, and other media purchases; and 3) paper and other supplies associated with advertising and recruitment. The monthly sum is adjusted (divided) by the mean number of open positions per month over the study period. The adjustment is made to take into account costs linked to open positions that may be difficult to separate out from costs linked to terminations. This figure is multiplied by the total number of terminations for the month and equals the monthly component cost of turnover linked to advertising and recruitment.

Vacancy costs
The sum of 1) labor and expenses linked to overtime due to understaffed conditions; and 2) missed or denied transport revenue due to understaffed conditions. The monthly sum is adjusted ( New-hire productivity costs The sum of the productivity costs for new hires (the difference between 90% productivity and productivity during the learning curve period). New-hire data were collected after the sixth month of data collection. The calculation for new-hire productivity appears in Appendix 1. All new-hire productivity costs were averaged and multiplied by the total number of terminations during the study period. This figure represents the total costs associated with bringing a new hire up to speed (90% productive) that can be linked to turnover. In situations where terminations exceed new hires, the total cost calculated cannot exceed the total of all expenses linked to new-hire productivity.

Preturnover productivity costs
The sum of labor costs linked to the time administrators or equivalent employees spend filling the shifts that employees who turnover during the month in question end up missing for one reason or another. The monthly sum is adjusted (divided) by the mean number of open positions per month over the study period. The adjustment is made to take into account labor costs linked to open positions and other staffing factors that may be difficult to separate out from costs linked to terminations. This figure is multiplied by the total number of terminations for the month and equals the monthly component cost of turnover linked to preturnover productivity costs.

Termination costs
The sum of 1) labor and expenses linked to the time administrators spend conducting exit interviews of terminated employees; 2) expenses associated with producing and printing materials and processing equipment and clothing linked to the termination process (e.g., washing or replacing employee agency-issued clothing and equipment); and 3) expenses linked to paying the terminated employee early retirement, unused vacation compensation, or other related payout expenses unrelated to providing him or her with a final check. contributors to the total cost of turnover. Among all-volunteer EMS agencies, the median total cost was $0 dollars across all seven categories of cost. However, it was determined that one of the three all-volunteer agencies had a projected annual vacancy related cost of $8,866.67. In table 5 we present the median cost per termination across EMS agency staffing mix. Figure 1 shows the agency-by-agency variation in median weeks and median costs associated with bring-ing newly hired EMTs and paramedics up to 90% productivity. Across all agencies with new hires during the study period (n = 27), the median number of weeks for a newly hired EMS employee to reach 90% productivity was 12 (Q1 = 8, Q3 = 18) and the median cost associated with new-hire productivity was $1,648.32 (Q1 = $242.00, Q3 = $3,458.26). Comparatively, the median number of weeks and the median cost associated with new-hire productivity were highest for all-paid EMS agencies and lowest for all-volunteer agencies. At the employee level across all new hires, the median number of weeks and the median total cost associated with new-hire productivity were higher for paramedic positions compared with EMT-B positions. Figure 2 illustrates that the median number of weeks and the median cost associated with bringing a newly hired paramedic up to 90% productivity were higher at allpaid EMS agencies compared with agencies with a mix of paid and volunteer staffing. Figure 2 excludes EMS agencies with all-volunteer staffing because of lack of newly hired paramedics during the study period at these agencies.

DISCUSSION
There are two known prior estimates of turnover in EMS. 13,28 These estimates suggest that turnover affects approximately one-fifth of EMS staff annually. However, the generalizability and validity of these findings are questionable. Both of these prior studies used a cross-sectional survey design. It is unclear whether the data collected accounted for fluctuations in turnover over time or how the rate of turnover in each agency was calculated. Variations in rates across agency demographics are not available. Additionally, the response rate for each administration was only 13.1% and 10.2%,  respectively. In our longitudinal study of 40 diverse EMS agencies, we determined that, on average, 10.7% of established EMS positions turnover annually. Findings suggest that the annual rate of turnover varies by agency staffing mix. The average annual rates of turnover for agencies with a mix of paid and volunteer staff and agencies with all-volunteer staff are slightly higher (12.3% and 12.4%) when compared with the average annual rate turnover among agencies with allpaid staffing (10.2%).
The variation in turnover observed in this study is not unique to EMS. Several studies of nursing turnover show wide variation across health care institutions and sectors. [29][30][31] Annual rates within sectors can also  It is therefore likely that the average rate of turnover and rates among agencies of different staffing configurations will vary from year to year. While it may require a substantial investment in time and resources, continuous monitoring of annual EMS turnover would ensure that the EMS community and the communities served stay informed of potential changes in the EMS workforce and possibly the services that agencies may provide under different staffing conditions. This study also examined the cost of EMS turnover, for which no prior estimates are known to exist. Modification of the NTCCM allowed us to quantify the total cost of turnover and costs per termination. Across agencies that experienced turnover, we determined that the total annual median agency cost of turnover was approximately $72,000. We observed wide variation in the total annual median cost of turnover across strata of agency staffing mix. Specifically, the total annual median cost of turnover was nearly nine times higher in agencies classified with all-paid staffing compared with agencies with a mix of paid and volunteer staffing. With the exception of one agency, agencies classified as having all-volunteer staffing experienced $0 costs during the study period.
The median cost per termination across all agencies that experienced turnover was $7,000 ( Table 5). The median cost per termination in all-paid staffing agencies was five times higher than among agencies with a mix of paid and volunteer staffing.
The studies by Jones on nursing turnover and associated costs show that the cost of nurse turnover in hospitals can be surprisingly high. In her most recent study, Jones determined that the total cost of nurse turnover was $8.5 million across three nurse service lines in a large acute care hospital. 22 The average cost per termination ranged from $82,000 to $88,000. 22 Compared with the costs identified in the current study, the costs associated with nursing turnover appear to be substantially higher than in EMS. However, direct comparisons between these two studies is tenuous, given differences in nurse and EMS employers, staffing models and configurations, educational backgrounds, and roles. The disparity in costs between this EMS study and studies in nursing implies that additional research is needed to better understand the costs and underlying causal mechanisms of turnover in each occupation.

LIMITATIONS AND FUTURE RESEARCH
There is potentially wide variation in EMS agency designs and models of delivery across the United States. We attempted to address the issue of sample representativeness by selecting and recruiting EMS agencies by using random selection and recruitment. However, this was limited by our sampling frame. The sampling was based on available data, and may influence the representativeness of our results.
Among those agencies receiving a recruitment packet, the longitudinal nature and time required by agency officials likely deterred many agencies from participating. Thus, a further limitation of our analysis is the low response rate. Turnover rates and cost calculations may be under-or overrepresentative of all of EMS depending on the patterns of turnover experienced in nonparticipating agencies. The limitation on representativeness of findings is particularly important when considering the all-volunteer EMS agencies studied. The all-volunteer EMS agency model varies greatly across U.S. EMS agencies, with some models paying employees based on transports, purchasing uniforms, and assuming the costs for other expenses related to our cost calculation methodology. For this reason, we believe that our findings most likely underestimate the overall costs of EMS turnover.
Several participating agencies had difficulty submitting their data on time at the end of each month. Where agencies lacked detailed records, tardiness with data submissions may have impacted the accuracy of responses to selected cost questions. Efforts to reduce tardiness included use of multiple e-mail reminders, telephone reminders, and on-site visits to help with data collection and submissions.
While our methodology for measuring turnover costs is based on previous research, and was pilottested extensively in the EMS setting prior to use in our full-study sample, this methodology may not capture all relevant turnover cost-related expenses relevant to the EMS setting. Further testing of our turnover cost calculation methodology may lead to new cost categories and new questions for measuring cost associated with turnover in the EMS setting. One next step for our study team includes collecting turnover and cost data from willing EMS agencies over multiple years. Other logical steps may include conducting qualitative studies of employees terminated and case studies of agencies with high and low rates of turnover and costs.

CONCLUSIONS
Turnover and the costs associated with turnover in EMS vary widely across types of EMS agencies. Based on our study findings, there is reason to believe that the annual rate of turnover may be slightly higher in EMS agencies that employ a mix of paid and volunteer personnel or all-volunteer personnel. The costs associated with turnover are highly variable across agencies and merit further investigation.