Development and psychometric evaluation of the Group Home Culture Scale.

Abstract Background Organizational culture in group homes for people with intellectual disabilities has been identified as influencing staff behaviour and residents’ quality of life (QOL). Despite this influence, culture has been under‐researched, with no published and validated instrument to measure its dimensions in group homes. The aim was to develop such a measure. Method The Group Home Culture Scale (GHCS) was developed using a theory‐driven approach. Items were generated from the research literature, which were reviewed by experts and tested in cognitive interviews. Data from 343 front‐line staff were used for exploratory factor analysis. Results The content and face validity of the GHCS were found to be acceptable. Exploratory factor analysis indicated that the GHCS measured seven dimensions of group home culture. Cronbach's alpha for the dimensions ranged from 0.81 to 0.92. Conclusions The GHCS has potential use in research to determine whether dimensions of group home culture predict the quality of staff support and residents’ QOL.

Adequate resources and settings of small size, for example, have been proposed to be necessary but not sufficient conditions for good outcomes (Bigby & Beadle-Brown, 2018;Emerson & Hatton, 1996). It has been suggested that organizational factors, such as how resources are used (Stancliffe, Emerson, & Lakin, 2004), management practices (Bigby & Beadle-Brown, 2018;Bigby, Bould, Iacono, & Beadle-Brown, 2019), and the way staff support is delivered and monitored may be critical (Emerson & Hatton, 1996). Organizational culture has been consistently identified as influencing service quality (Felce, Lowe, & Jones, 2002;Hastings, Remington, & Hatton, 1995;Walsh et al., 2010) based on the assumption that culture influences staff behaviour, and in turn, residents' QOL.
The notion that culture influences staff behaviour has been evident in the wider organizational literature since the concept came to prominence in the 1980s (Ehrhart, Schneider, & Macey, 2014;Smircich, 1983). Many definitions of organizational culture have appeared in the literature (Verbeke, Volgering, & Hessels, 1998).
For example, it has been broadly defined as "the way we do things around here" (Deal & Kennedy, 1982, p. 4) and more comprehensively defined by Schein (2010) as …a pattern of shared basic assumptions learned by a group as it solved its problems of external adaptation and internal integration, which has worked well enough to be considered valid and, therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to those problems. (p. 18) Culture has also been conceptualized as existing at multiple levels, for example at the broader organizational and staff group (or team) levels, with potentially numerous subcultures within organizations (Martin, 2002;Trice & Beyer, 1993). Despite differing conceptualizations, key concepts of culture include shared beliefs, values, norms, assumptions, ideologies and meanings (Alvesson, 2013;Ehrhart et al., 2014).
The conceptualization of culture frequently discussed in the intellectual disability services literature has included a distinction between formal and informal organizational culture (Emerson, Hastings, & McGill, 1994;Felce et al., 2002;Hastings et al., 1995;. Formal culture refers to operational policies, job descriptions, planning systems, working methods, training and mechanisms used to monitor staff (Felce et al., 2002). Arguably, they are aspects that are controlled by management and designed to influence staff behaviour through specifying expectations and constraining what staff do (Felce et al., 2002). Informal culture, on the other hand, refers to ways of working as defined by the staff group, interactions and relationships among staff, and also between staff and residents, and reasons for working (Felce et al., 2002;Hastings et al., 1995). For more than 20 years, congruence between formal and informal culture has been argued to contribute to higher quality services than incongruence (Emerson et al., 1994), but research has not been conducted to confirm this proposition.
There have been few studies into organizational culture in group homes, or more generally supported accommodation services. In one such study, Gillett and Stenfert-Kroese (2003) examined organizational culture and residents' QOL in two residential units from the same organization using the Organizational Culture Inventory (OCI; Cooke & Lafferty, 1989, as cited in Gillett & Stenfert-Kroese, 2003).
They found that residents in the unit with the more positive culture also had higher QOL. The OCI is a generic instrument that measures organizational culture in terms of behavioural norms (Cooke & Szumal, 2000), and has been used by researchers in a wide variety of organizations (Kummerow & Kirby, 2014). It comprises 120 items, and although some appear applicable to supported accommodation services (e.g. staff members "help others to grow and develop" and "do what is expected"), others appear less relevant (e.g. staff members "turn the job into a contest" and "use the authority of their position"; Balthazard, Cooke, & Potter, 2006, p. 720).
In another study applying a generic measure, Hatton et al. (1999) used the Organizational Culture Profile (OCP; O'Reilly, Chatman, & Caldwell, 1991) to examine associations between culture and staff outcomes (e.g. job satisfaction, commitment) in services for people with intellectual disabilities. The OCP assesses values through 54 statements. Although some statements arguably are relevant to supported accommodation services (e.g. "being people oriented" and "enthusiasm for the job"), some are less so (e.g. "high pay for good performance", "being competitive" and "being aggressive"; O'Reilly et al., 1991, p. 516).
These two generic measures, the OCI and OCP, were not designed for group homes. As such, they do not measure culture relating to interactions between staff and residents of group homes and are mostly about interactions among staff and employment conditions.
The limited research into culture in group homes for people with intellectual disabilities has primarily used qualitative methodologies.
Using an ethnographic approach, and Schein's (2010) definition of culture, Bigby and colleagues (Bigby & Beadle-Brown, 2016;Bigby, Knox, Beadle-Brown, & Clement, 2015;Bigby, Knox, Beadle-Brown, Clement, & Mansell, 2012) conducted two studies in five underperforming and three better performing group homes for people with severe intellectual disabilities. They identified five dimensions of culture: (a) alignment of power holders' values, (b) regard for residents, (c) perceived purpose, (d) working practices and (e) orientation to change and new ideas. Each dimension was proposed as a continuum from negative, which was more likely to apply to underperforming group homes, to positive, which was more likely to apply to high performing group homes.

HUMPHREYS Et al.
Comparisons of the culture across the underperforming and better performing group homes in these studies demonstrated that the way staff worked and interacted with the residents were markedly different. For instance, staff practices in the underperforming group homes were characterized as staff-centred, prioritizing completion of tasks and interactions that involved staff doing things for or to residents. In contrast, staff practices in the better performing group homes were characterized as person-centred, whereby relationships between staff and residents were described as warm, with interactions that included moments of fun. Furthermore, Bigby, Knox, Beadle-Brown, and Bould (2014) showed that the residents who lived in the group homes with more positive cultures had higher QOL. Bigby and colleagues (Bigby & Beadle-Brown, 2016;Bigby, Knox, et al., 2012) suggested that the dimensions they identified could provide a basis to develop a quantitative measure of organizational culture in group homes, with potential advantages over the generic instruments used to date. They further argued that such an instrument would be more likely to point to relevant implications for service delivery, and be more translatable into strategies to improve culture than generic instruments.
This study aimed to develop the type of instrument for measuring culture in group homes suggested by Bigby and colleagues (Bigby & Beadle-Brown, 2016;Bigby, Knox, et al., 2012). The definition of organizational culture used required concepts that could be more readily measured using quantitative methods, than those in Schein's (2010) definition, such as staff members' assumptions, which require intensive observations and interviews to be identified. Accordingly, culture was defined using features described across the broad organizational culture literature, and then those pertinent to group homes for people with intellectual disability, as staff members' shared values, beliefs, norms and patterns of behaviour that influence how they think, feel and act (Ott, 1989;Trice & Beyer, 1993). The purpose of this study was to develop and evaluate the psychometric properties of an instrument to measure dimensions of organizational culture in group homes-named the Group Home Culture Scale (GHCS). The GHCS was designed as a self-report instrument to be completed by disability support workers (DSWs) and front-line supervisors, who are variously known as house supervisors, team leaders or service managers.

| Design
A mixed-methods sequential research design (Creswell & Plano Clark, 2011) was used with the following stages (a) item development, (b) expert review, (c) cognitive interviews and (d) questionnaire administration. Quantitative and qualitative data were collected and analysed to refine the conceptualization and measurement of constructs, as well as to assess the content validity, face validity and internal consistency of the GHCS. Ethics approval for this study was provided by the La Trobe University Human Research Ethics Committee.

| Item development
Development of the GHCS was theory-driven (DeVellis, 2012;Hinkin, 1998;Wymer & Alves, 2013), based on the five dimensions of group home culture proposed by Bigby and colleagues (Bigby & Beadle-Brown, 2016;Bigby, Knox, et al., 2012), and research examining culture in supported accommodation services where allegations of staff abuse has occurred (Cambridge, 1999;Marsland, Oakes, & White, 2007). Drawing on this literature, the content domain of each dimension was specified (i.e. the concepts and elements that comprised each dimension) and then items were written to tap each dimension (DeVellis, 2012). Items were written to tap both ends of the continuum (i.e. positive and negative) for each dimension, and to adhere to item writing guidelines, such as simple and direct language be used, and each item conveys a single idea (Gideon, 2012). Consistent with the referent shift consensus composition model, items were phrased so that respondents reflected on their staff group (or team) and their shared, rather than individual ways of working (Chan, 1998;Klein, Conn, Smith, & Sorra, 2001). For example, the statement "Staff plan with residents what happens on weekends" has a group referent, in contrast to "I plan with residents what happens on weekends," which has an individual referent.
A large pool of items (n = 359) was generated by the first author, which was reviewed and refined by the research team over several meetings. Items that tapped the dimensions were identified, others were either revised or discarded, and new items were generated.

| Expert reviews
The initial 197 items were reviewed by experts, defined as academics who had published research on supported accommodation services.
Twelve experts, identified using purposive sampling, were sent information via email about the study and invited to participate. Those who expressed interest were sent further information, including the definition of organizational culture, conceptual descriptions for each dimension, and the list of items which they were asked to rate on a 4-point Likert scale (1 = not representative to 4 = completely representative) according to the extent to which each item measured the dimension and its clarity (1 = not clear to 4 = very clear). Written feedback was also sought regarding the clarity of the items, suggestions for improved phrasing or new items, whether any items should be deleted or moved to another dimension, and comments on the conceptualization of the dimensions.
Four reviews were returned. Two experts rated all the items, one rated the representativeness for most (82%), and another rated the items inconsistently, but provided comments for many. The Content Validity Index (CVI; Lynn, 1986;Polit & Beck, 2006) was used to analyse the representativeness ratings for each item by calculating the proportion of experts that assigned it a rating of 3 or 4. Lynn (1986) recommended a CVI of 1.0 to retain an item when there are five or fewer participants. Using the ratings from three experts resulted in 82 items obtaining a CVI of 1.0 (CVI range = 0.33-1.0). The CVI was then calculated for items rated by only two experts, which resulted in an additional 30 items obtaining a CVI of 1.0. However, there were two problems with using only the CVI to retain items: (a) the retained items may not have tapped the content domains of the dimensions comprehensively; and (b) the qualitative data were not considered. To address these problems, the retained items were mapped according to the content domains for each dimension. For those content domains that were not comprehensively tapped, the suitability of the items with a CVI of less than 1.0 was assessed. This process involved tallying the representativeness ratings provided by three experts (range = 3-12), and items with a total score of 9 or 10 were assessed as to whether they should be retained based on a review of the written comments and clarity ratings. Eleven items were retained using this process, some of which were then revised.
Over several meetings, the retained items (n = 123) were reviewed by the research team, and the comments and suggestions made by experts were discussed. Following this process, 41 new or revised items were added, bringing the total to 164 items.

| Cognitive interviews
To test the items, 16 cognitive interviews (Willis, 2005) were conducted with 15 participants who had experience of working in group homes, recruited using convenience and snowball sampling. Once they provided written consent, they were emailed a questionnaire that comprised the items and asked to complete it either before or during the interview. Telephone interviews were conducted, most of which were approximately 1 hr in duration (range = 20-80 min), and handwritten notes were made. One participant who had extensive experience as a disability support worker was interviewed twice, in separate rounds, because he provided significant insight into the cognitive process of interpreting and answering items. Participants were asked how they understood and interpreted items, how they formulated answers, and to describe the experiences they recalled to answer questions. They were also asked about the design of the questionnaire, such as the layout, instructions and the response format.
Following Willis' (2005) recommendation, interviews were conducted in rounds, with data analysed between them, until all major problems had been corrected. After each round, data were analysed by producing summaries and making comparisons across participants (Miller, Willson, Chepp, & Ryan, 2014). Items that were consistently understood as intended were retained; items that were problematic or ambiguous were either dropped from the questionnaire, or revised and retested in subsequent interviews. In this way, the face validity of the GHCS was assessed. After four rounds of interviews, 86 items remained.

| Questionnaire administration for exploratory factor analysis
In order to identify the underlying factor structure and evaluate the internal consistency of the GHCS, a questionnaire comprising the 86 items was administered to DSWs and front-line supervisors who worked in group homes for people with intellectual disabilities.

Nongovernment intellectual disability organizations that operated in
Australia were approached via email and managers from 10 organizations agreed to participate in the research. These organizations varied in location and size, operating in three states (New South Wales, Victoria and Western Australia) in both metropolitan and nonmetropolitan areas, and managing from 5 to 31 group homes (Mdn = 9).
Inclusion criteria were DSWs and front-line supervisors who worked in 24 hr staffed group homes in which up to eight adults with intellectual disabilities were supported. Staff were excluded if they had worked in the group homes for <2 months and/or worked, on average, <4 hr per week.

| Participants
Questionnaires were completed by 380 staff, representing an overall response rate of 43% (range = 8%-64% across organizations). Of the respondents, 343 (279 DSWs and 64 front-line supervisors) met the eligibility criteria and provided data that were usable for exploratory factor analysis (EFA).
Participants were on average 44.4 years of age (SD = 13.2, range = 20-74), 68.7% were female and 57.4% born in Australia. As shown in Table 1, over half of the participants had more than 3 years' experience of working in accommodation services. Most participants worked, on average, 26 hr or more per week in the group homes. The mean number of residents per group home was 5.1 (SD = 1.1, range = 2-8).

| Group Home Culture Scale (GHCS)
Respondents were asked to rate each of the 86 items on a 5-point Likert scale anchored by strongly disagree (1) and strongly agree (5). The first page contained instructions on how to complete the measure and definitions of key terms (e.g. house supervisor, team, senior managers). Nine items on the DSW and front-line supervisor versions of the GHCS differed: front-line supervisors were asked to self-report about their own leadership, in which case, the referent was themselves; DSWs responded to corresponding items in terms of the leadership of their front-line supervisor.

| Demographic and employment information
Demographic and employment information about respondents was obtained from closed questions (n = 13) at the end of the GHCS. These questions addressed gender, age, country of birth, level of education, employment experience in disability accommodation services, and employment experience and average hr per week worked in the group home.

| General procedures
Once written consent from a manager at each organization had been obtained, group homes that met the eligibility criterion were identified. Questionnaire packets were posted to the manager or contact person, who then distributed them to front-line supervisors and DSWs. Completed questionnaires were returned to the research team in provided prepaid envelopes. Data were collected from October 2015 to February 2016.

| Analyses
Data were entered and analysed using SPSS 22. Descriptive statistics for the sample were calculated. EFA was conducted for the 86 items GHCS to identify the underlying structure among the variables. EFA was chosen instead of confirmatory factor analysis because the GHCS was a new measure, and although the dimensions and items were developed from theory, the number of factors and their composition were uncertain (Fabrigar, Wegener, MacCallum, & Strahan, 1999).
Prior to EFA, an analysis of missing data was performed, showing missing data for 80 (22.5%) of 356 GHCS questionnaires. Of these questionnaires, those for which there was more than 15% missing data (n = 13) were removed from further analysis. Of the remaining 343 questionnaires, there were missing data on 67 (19.5%) for 1-9 items. Expectation maximization was used to impute missing data (Graham, 2009). Negatively phrased items were reverse scored.  (Hair, Black, Babin, & Anderson, 2014;Tabachnick & Fidell, 2014). Principal axis factoring, which accounts for common variance, was used to extract the factors (Hair et al., 2014). The number of factors to retain was determined by examining eigenvalues and the scree plot, and conducting a parallel analysis. Of these factor retention methods, parallel analysis has been recommended as the most accurate (Zwick & Velicer, 1986). The parallel analysis was performed by comparing the eigenvalues to those obtained from randomly generated datasets based on the same sample size (Pallant, 2013). The number of factors to retain was indicated by the eigenvalues exceeding those that were randomly generated. An oblique (direct oblimin) rotation was performed to allow the factors to correlate. A minimum factor loading of 0.40 on the pattern matrix was used to retain items. The internal consistency of the factors was assessed using Cronbach's alpha. Because Cronbach's alpha has been shown to underestimate the true level of internal consistency when the assumption of tau equivalence is violated (McNeish, 2018), the greatest lower bound (GLB) was also used to assess internal consistency. GLB was calculated using the program JASP 0.9 (JASP Team, 2018).   (Hair et al., 2014).

| RE SULTS
Following principal axis factoring, application of the Kaiser criterion indicated that 16 factors be retained. The scree plot did not provide a clear indication of the number of factors to retain. Rather, the parallel analysis indicated that seven factors had eigenvalues exceeding those randomly generated; hence, seven factors were retained and rotated using the direct oblimin method. Finally, following an examination of factor loadings and communalities, 48 items were retained.
The GHCS subscales were named based on an examination of the items with high loadings on each factor (Hair et al., 2014) and with reference to the original conceptualizations of the dimensions (Pett et al., 2003). The seven factors, their names and descriptions are presented in Table 2.  (Cohen, 1992), and that they represent related but also distinct dimensions of group home culture. Table 4 also shows descriptive statistics and sum of squared structure loadings for each factor.

| D ISCUSS I ON
This study has resulted in the development of an instrument to measure dimensions of organizational culture in group homes, named the Group Home Culture Scale (GHCS). It contrasts to the generic instruments used in previous research of intellectual disability services (Gillett & Stenfert-Kroese, 2003;Hatton et al., 1999)  were misaligned with those espoused by the organization (Bigby, Knox, et al., 2012). On the other hand, in another underperforming group home, power was more dispersed, with each staff member adopting his or her own way of working (Bigby, Knox, et al., 2012).
In contrast, Bigby and Beadle-Brown (2016) found that in each of the better performing group homes, the formally appointed power An unexpected result was the failure of the GHCS to include as a dimension staff resistance or openness to change and new ideas (Bigby & Beadle-Brown, 2016;Bigby, Knox, et al., 2012). This result may be explained by small intercorrelations and factor loadings among some of the items written to tap this content domain, indicating that they did not comprise a common underlying dimension of group home culture. In part, problems with these items could reflect difficulties for respondents in rating them consistently because how staff respond to change and new ideas can also be inconsistent.

TA B L E 3 (Continued)
group homes as resistance to change and new ideas, but some of their evidence suggested that some staff, rather than all, were exhibiting resistance. On the other hand, Bigby and Beadle-Brown (2016) characterized the culture in the better performing group homes as openness to change and new ideas, but, as they noted, a family member's suggestion about how to decorate a resident's room was opposed by a staff member because it was thought to be inconsistent with the resident's preferences. It would appear, then, that context is a potential factor when considering how staff respond to new ideas.
A limitation of this study was that the GHCS was developed based on research of underperforming (Bigby, Knox, et al., 2012) and better performing group homes (Bigby & Beadle-Brown, 2016;Bigby et al., 2015), instead of those considered to be of high quality. In general, there has been a lack of research into culture in high-quality group homes, and rather, more is known about the culture in poor quality and abusive services (see Bigby, Knox, et al., 2012;Cambridge, 1999;Hutchison & Stenfert-Kroese, 2015;Marsland et al., 2007).
To enhance the GHCS, new items could be generated that better reflect the culture in high-quality services; however, qualitative research that explicates the characteristics of these services may first be required. Another limitation of this study was that the level of adaptive behaviour of the people who lived in the group homes was not measured, which meant that the potential effect of resident level of adaptive behaviour on staff member ratings of GHCS items was not assessed. Future research is needed to determine any potential relationship between resident adaptive behaviour and GHCS scores.
Of relevance to practice is that the GHCS can be used by organizations to measure staff perceptions of their work culture.
Information collected with the GHCS has potential to be used by organizations to understand culture in group homes and identify opportunities to improve it. Potential advantages of using the GHCS in group homes over a generic instrument is that the findings could have clearer implications for service delivery and be more translatable into strategies to change or maintain culture.
With the use of the GHCS, there is the potential for future research into identifying the dimensions of organizational culture that are associated with the quality of staff support and QOL outcomes for people with intellectual disabilities who live in group homes.
The GHCS also has potential use in research to examine whether dimensions of group home culture are associated with staff outcomes, such as job satisfaction. To test the factor structure of the GHCS, further research is needed using confirmatory factor analysis.