Explicit and Implicit Measures of Expectancy and Related Alcohol Cognitions: A Meta-Analytic Comparison
Abstract
Implicit measures assess the influence of past experience on present behavior in the absence of respondents’ awareness of that influence. Application of implicit measurement to expectancy and related alcohol cognition research has helped elucidate the links between alcohol-related experiences, the functioning of alcohol-related memory, and alcohol-related behavior. Despite these advances, a coherent picture of the role of implicit measurement has been difficult to achieve due to the diversity of implicit measures used. Two central questions have emerged: do implicit measures assess a distinct aspect of the alcohol associative memory domain not accessible via explicit measurement; and, when compared to explicit measurement, do they offer unique prediction of alcohol consumption? To the end of addressing these questions, a meta-analysis of studies using both implicit and explicit measures of alcohol expectancy and other types of alcohol-related cognition is conducted. Results indicate that implicit and explicit measures are weakly related, and while they predict some shared variance in drinking, each also contributes a unique component. Results are discussed in the context of the theoretical distinction made between the two types of measures.
Explicit and Implicit Measures of Expectancy and Related Alcohol Associations: A Meta-Analytic Comparison
In the field of alcohol research, expectancies measured via questionnaires have been well supported as mediators of the relationship between other known antecedents of drinking and eventual consumption (Goldman, Del Boca, & Darkes, 1999). This particular use of expectancy/anticipation as an explanatory device fits, however, within a much broader and growing expectancy literature that includes such widely diverse phenomena as animal reward and reinforcement (Kupfermann, Kandel, & Iversen, 2000; Schultz, Dayan, & Montague, 1997; Schultz, 2004), perception of motion (Kerzel, 2005), development of language (Colunga & Smith, 2005), and time perception (Correa, Lupiáñez, & Tudela, 2005), among many others (see Goldman, 2002; Goldman, Reich, & Darkes, 2006).
It is apparent that much of this broader expectancy/anticipation literature is not confined to examining explicit processes, but explores what would fall well within the domain that is considered implicit in nature. It follows, therefore, that recent alcohol expectancy research also would investigate “implicit” expectancy processes. The purpose of this line of work has been to better understand those decision-making processes that are not accessible to consciousness, and particularly the degree to which decision-making is influenced by ongoing changes in context absent awareness of this influence (see De Houwer, 2006). The presence of both explicit and implicit expectancy processes is consistent with theories of alcohol-related memory; alcohol expectancies may be understood, in part, as associative memory links between alcohol and the consequences of its use, including cognitive, affective, and behavioral outcomes (Goldman, Darkes, Reich, & Brandon, 2006), that permit organisms to anticipate/predict reward or punishment following behavioral actions (Schultz, 2004). The relationship between explicit and implicit expectancy processes in the alcohol domain as measured by currently used instruments is as yet uncertain, however. To help clarify this relationship, a meta-analysis of currently available studies of alcohol expectancies that include both explicit and implicit measures could be informative.
Implicit Measures of Cognition
To better inform this domain-specific application of the explicit-implicit distinction, and to be consistent with recent recommendations for thorough familiarity with the relevant body of literature before conducting a meta-analysis (Rosenthal & DiMatteo, 2001), we first briefly review general issues related to the study of implicit cognition:
In the 1980's and 90's, a wave of indirect measures was developed by researchers examining general memory processes to further explore a paradox observed in some amnesiacs: that memory performance can be influenced by previous experience in the absence of direct recall of that experience, or even awareness that such experience may be influencing memory (e.g. Tower of Hanoi, mirror tracing; Jacoby, 1991; Roediger, 1990; Schacter, 1987; Willingham & Preuss, 1995; Roediger, 2003). Although this general characteristic is central to all implicit memory tasks, specific implicit tasks may vary greatly in terms of the nature of the previous experience that influences performance and as to the memory requirements of the task. Experiences that may implicitly influence performance may be very recent (and may last as briefly as milliseconds), or more distant (e.g., discrete or accumulated life events that are not necessarily specific to the task, but nevertheless may influence performance on that task). Memory requirements of the task may be recall, recognition, or free associate production; completion of word fragments or stems; or speed (latency) to recognize or categorize words (Roediger & Geraci, 2005).
Distinguishing Implicit Process from Explicit Processes
Despite what may appear to be new and distinctive findings offered by the use of implicit methods, debate over how to classify a task as implicit or explicit continues both in cognitive psychology (Roediger, 2003), and in other fields to which implicit methods have been applied (e.g., see Fazio & Olson, 2003). Furthermore, theoretical agreement has not been reached in any field regarding the underlying process(es) corresponding to implicit and explicit measurement (i.e., are they due to different task requirements or to different memory “systems;” do one, two, or more memory systems exist; do distinctive neuroanatomical or neurophysiological underpinnings provide the substrate for explicit vs. implicit tasks; are explicit and implicit domains non-overlapping; etc?; see Roediger, 2003; Ryan & Cohen, 2003, for different perspectives). Cognitive researchers continue to debate the extent to which various memory tests have conscious or unconscious components and the extent to which memory is actually composed of separate implicit and explicit systems. These arguments have yet to be resolved, but it is clear that the preponderance of evidence does not support binary either/or approaches (Roediger, 2003; Roediger, Buckner, & McDermott, 1999; Gawronski, LeBel, & Peters, 2007; Fazio & Olson, 2003). And even at a theoretical level, Reder, Park , and Kieffaber (2009) have recently pointed out that even the dissociations found between implicit and explicit tasks, typically viewed as the ultimate evidence for distinct underlying memory systems, could be explained using a single memory system. They posit a memory system that stores different kinds of information, including information about an episode to which other memory relevant information may or may not be “bound.” If bound to episodic information, the memory operates explicitly; if not bound to an episode, the memory appears implicit. Performance on different tasks requires activation of different binding patterns. (Because ample evidence exists of distinctive memory activation by implicit and explicit tasks, meta-analysis of the sort presented in this paper remains important. From the Reder et al. [2009] perspective, however, the meta-analytic results reported herein would bear only upon task distinctiveness and utility, but would not help distinguish between theories of memory operation. This point is revisited in the discussion section.)
At the measurement level, a core obstacle to addressing these questions is the difficulty that has been encountered in determining the “purity” of implicitness of measures. As Roediger (2003) pointed out, most purportedly explicit psychological measures (including paper and pencil surveys) could be considered to have an implicit component in that previous events almost always prime responses in the absence of the respondent's full awareness of their influence. Consider the extensive array of alcohol expectancy measurement devices that are influenced by past drinking (and other) experiences. Although individuals may be likely to remember some or all of these experiences, complete awareness of how these experiences influence their answers or patterns of answers on a questionnaire is unlikely. By the extant definitions, therefore, these measures would be considered at least partially “implicit,” although they have not been customarily so regarded. As just one example of this kind of influence, Weinberger, Darkes, Del Boca, and Goldman (2006) showed that the order in which items are presented in a questionnaire influences the obtained factor structure. These organizational shifts clearly occur outside of respondent's awareness. To respond to explicit instruments, respondents may deliberately recollect a specific experience, or they may simply go with their subjective feeling or impression to guide their choice. How the latter strategy would differ from implicit processing is difficult to specify.
Viewed from the opposite vantage point, it is equally difficult to demonstrate that measures specifically designated to be implicit are free of explicit influences (Gawronski, LeBel, & Peters, 2007; MacDonald, 2008). Fazio and Olson (2003) describe a body of evidence that demonstrates that people can “correct” for implicitly activated attitudes in socially desirable ways. That is, their behavior is not necessarily consonant with their attitudes evidenced using implicit tasks. That cognitions can be influenced by social desirability not only implies awareness by the respondent of the attitudes being measured, but also indicates that to some extent respondents can control their “implicit” (and supposedly automatic) output. The presence of such control brings into question the extent to which these outputs are exclusively automatic. As we now turn to the application of implicit measures to the alcohol domain, these continuing debates must be kept in mind as we investigate the relative contribution of implicit and explicit measures to the prediction of drinking.
Implicit Measures of Alcohol Expectancy and Related Alcohol Cognition
Examination of the implicit alcohol expectancy literature shows these measures to fit within a much broader and growing literature that characterizes implicit decision-making processes about alcohol and other substance use in a variety of ways (see Goldman, 2002; Goldman, Reich, & Darkes, 2006). This expanded perspective is well characterized in two recent books, in which researchers have offered a number of different models of the relationship between explicit and implicit processes involved in substance use behavior (Wiers & Stacy, 2006; Munafò & Albery, 2006). This body of work makes clear that, despite differing theoretical origins, studies of explicit and implicit cognitive constructs other than “expectancy” also might provide useable data for the present analyses because they are sufficiently similar at the operational/measurement level. That is, understood as associative memories, many findings from the alcohol expectancy literature can be interwoven with other themes in alcohol research that relate cognition to decision-making (see Stacy, Ames, & Grenard, 2006; Cox, Klinger, & Fadardi, 2006).
Such a related theme emerges from the theoretical framework referred to as “attitudes.” Attitudes, like expectancies, fall within the domain of cognitive processes that have been studied extensively using implicit measures; first in social psychology (Greenwald & Banaji, 1995), and more recently in substance use-related research (De Houwer, 2006). To illustrate the overlap, words such as “sociable,” “pleasant,” “good,” “bad,” or “relaxed,” (among many others) may be found on measures of expectancies and attitudes. For the purposes of this study, attitudes and expectancies have been assessed using the same types of implicit measures (e.g., the Implicit Association Test; IAT). The distinction between expectancies and attitudes most typically rests on a temporal dimension, with expectancies reflecting anticipation of a future outcome (an if–then relationship), and attitudes reflecting an evaluative memory association irrespective of timing. This difference can be bridged in the present context (a meta-analysis of measurement instruments) because measures of both constructs assess substance use and its associates (an anticipated consequence or an evaluation) concurrently. And bridging these perspectives (by including studies from these other domains) would not only enhance the reliability of the present meta-analysis by increasing the number of included studies, but also would extend the generalizability of the results; increased generalizability may be considered an ultimate purpose of meta-analysis. Therefore, although we began with the intent to perform a meta-analysis of explicit and implicit alcohol expectancies exclusively, we evolved toward an expanded meta-analysis that provides feedback on the present status of a broader array of explicit and implicit alcohol cognitions.
Meta-analysis of all studies that examine explicit or implicit alcohol cognitions separately would necessitate, however, inclusion of an extremely large array of investigations. Explicit studies of alcohol cognitions alone now number into the thousands. And, underscoring the increasing availability of implicit studies of alcohol cognitions, a recent meta-analysis by Rooke, Hine, and Thorsteinsson (2008) identifies 48 studies that use implicit measures. Although Rooke et al. (2008) find several moderators of effect size between these implicit measures and alcohol use, they conclude that overall, implicit techniques reliably measure such cognitions, albeit with a small mean effect size of .23. Examples of the tasks identified include the Stroop, false memory, free association, extrinsic affective Simon task (EAST), and the implicit association test (IAT). These studies consistently show that scores on implicit tasks are related to reports of drinking; heavier drinkers associate positively and/or negatively valenced effects of alcohol when primed with alcohol cues (the strongest associations are with positively valenced effects), and in general, lighter drinkers more strongly associate negatively valenced and/or sedating effects with alcohol cues.
The present meta-analytic review has a different purpose: to directly compare implicit approaches (of the kind reviewed by Rooke et al., 2008) to explicit approaches using only studies that use both implicit and explicit tasks with the same sample of participants (a small fraction of the Rooke at al. [2008] “implicit” studies fall into this category and are added to other available studies in the present meta-analysis). Two primary questions are addressed: do implicit measures assess a distinct aspect of the alcohol cognition domain not accessible via explicit measurement, or, alternatively, do they just offer a new set of indices of the same alcohol-related cognitive constructs defined previously by explicit measures? Also, at a practical level, could prediction of drinking using alcohol-related cognitive constructs be improved by combining implicit and explicit measures (put another way, do implicit expectancy measures add uniqueness in the prediction of drinking beyond that contributed by explicit measures)? The decision to confine the studies identified for this meta-analysis to those that include both types of measures in the same study stems from the intent to answer these questions. This strategy allows us to make comparisons using a repeated measures approach, thereby reducing variation due to sample differences (increasing reliability of the comparison).
This study is designed to be exploratory in nature, and thus is agnostic as to the eventual outcomes. Conceptual review of the relevant research in this and related domains that have used implicit and explicit approaches (e.g., social psychology, smoking) could suggest specific hypotheses, however. This previous work has shown a small degree of relatedness between implicit and explicit measures and an advantage for explicit measures over implicit measures in prediction of behavioral outcomes. We anticipate, therefore, a similar pattern of results in the present meta-analysis.
Method
Process of Selecting Studies for Meta-analytic Review
Selection is carried out using the following keywords (either individually or as combinations) in the Psycinfo database: Alcohol, Alcohol Expectancy, Implicit Memory, Implicit Cognition, Priming, Attitudes, and Memory Associations. As noted in the introduction, initial attempts to conduct this meta-analysis using only studies labeled “alcohol expectancy” yielded too few studies to conduct a reliable meta-analysis. We therefore expand our inclusion rules to add studies of “attitudes” and more generally, “implicit cognitions.” Although these constructs cannot be considered theoretically the same as alcohol expectancies, as with expectancies, the processes through which they are presumed to exert influence include associative memory links between alcohol and domain-relevant cognitive, affective, and behavioral information. As a result, the operational overlap is sufficiently great that the items used in implicit measures of alcohol expectancies and the items used in implicit measures of these other domains can be difficult to distinguish. (Aspects of each of these theoretical approaches have already, or may in the future, suggest measurement operations that do not overlap; we confine ourselves here to existing implicit memory techniques.)
Of studies so identified by these broader criteria, only those that contain an implicit and an explicit measure of alcohol-related cognitions along with a measure of drinking are retained. Because participant age group is identified as a moderator in the Rooke et al. (2008) meta-analysis, and too few studies of adolescents include both implicit and explicit measures, only studies of adults are included. Virtually all identified studies use college undergraduates (mean age across studies was 20.71). Sixteen studies, with a total n of 1857 participants, remain for final review and analysis. All identified studies with a publication date through 2006 overlap with those in the Rooke et al. (2008) meta-analysis. (Studies with a 2007 or 2008 publication date are likely too recent to be included their study.)
During individual evaluation, these sixteen studies are compared on operationalization of relevant variables, procedures, and statistical analyses. Particular attention is paid to the degree of statistical relation (i.e., Pearson Product Moment Correlations) reported between each type of expectancy measure and between each expectancy measure and drinking. These correlations become the units of measurement in meta-analysis. When the statistics relevant to this review do not appear in an article, the first author is contacted via Email and requested to send the missing statistics.
This review is complicated by the use of different drinking variables as dependent measures in the selected studies, precluding the inclusion of a unified metric of drinking. These variables include drinking quantity and frequency, episodes of heavy (binge) drinking, and problems associated with drinking. Although these variables are different, they are typically significantly correlated with one another. Despite this overlap, however, some variation in the drinking outcome variables used in the meta-analysis has to be accommodated. To the extent possible, of course, common variables are included. As can be seen in Table 1, most effects included in the analysis use quantity, frequency, or composites including both quantity and frequency, as the measure of drinking. As a safeguard to rule out the possibility that different drinking outcomes would lead to different results, two moderator analyses are conducted to exclusively examine the prediction of frequency and quantity of drinking (separately), using those studies reporting such statistics. Summary data for the selected studies, including effect sizes, can be found in Table 1.
Statistical Aggregation of the Selected Studies
To determine possible differences between implicit and explicit alcohol expectancy measures in accounting for drinking variance, mean effect sizes are calculated for the relationship between implicit and explicit measures, implicit and drinking measures, and explicit and drinking measures. Because most studies include several measures of drinking outcomes and multiple subscales within their explicit and implicit cognitive measures, each study reports several correlation values (effect sizes). Two approaches are considered to address the multiple effects: 1) Take the average effect from each study; or 2) Take the largest effect as the index effect size from each study. Because the central purposes of this study are to evaluate the capabilities of explicit and implicit cognitive measures to predict and explain variance in drinking, and to compare the two types of measures, the largest effect from each study is chosen as the index effect size.1 To decrease the possibility that choosing the largest effect would lead to some kind of bias in the meta-analysis, a moderator analysis is also conducted evaluating the prediction of multiple drinking outcomes separately.
To normalize the distribution of effects reported for each individual study (r), these effects initially are transformed using Fisher's Z transformation. Because of the high variability of n between studies (ranging from 32 to 342), transformed Z scores are then weighted by n-3 for each study (the convention for meta-analysis). Mean unweighted and weighted Z effects then are calculated and converted back to r to be reported as the overall effect size.
Because one primary interest of this study is to compare unique prediction of drinking by implicit and explicit measures, Beta weights also are used as a measure of effect size. Because beta weights are sometimes poor measures of effect size in regression models because they may over-, or underestimate the magnitude of the predictor-criterion relationship depending on the other variables included (Rosenthal & DiMatteo, 2001), effect sizes of r (noted above) are calculated along with the β effect sizes.
A major feature of this review is the statistical comparison of implicit and explicit effects. Essentially, we conduct two meta-analyses; one for implicit effects and one for explicit effects and then compare the two. Because type of measure (implicit vs. explicit) is a within-study factor, mean effects (β and r) for the implicit/drinking and explicit/drinking relationships are compared statistically using a paired samples approach. Due to T-test's assumptions of normality, we intended to use the z-transformed β and r means for this comparison. Because the Z transformation has little effect on small or moderate correlations (as are most of the effects observed here), the distribution of effects remain highly skewed (1.39 < skew < 3.39). As a result, the Wilcoxon Signed Ranks Test, a non-parametric test of paired samples, is the statistical test used for these comparisons.
Results
Review of Specific Studies Measuring Alcohol Expectancies Implicitly and Explicitly
Before conducting the meta-analytic statistical procedure with all of the studies grouped together, first the studies are evaluated individually. This evaluation serves to 1) ensure that the studies are sufficiently similar, in terms of operational definitions and procedures, to permit treating them as from the same population of studies, and 2) provide a qualitative review of the nuances of each study, such as unique methods of measuring the same construct, along with a qualitative comparison of the results of each compared with the others.
Free association (FA)
Stacy (1997) uses an FA task (implicit) and a participant-generated expectancy scale (explicit) to predict alcohol use in a prospective study. In FA tasks, participants are asked to produce responses to written prompts as quickly as possible. Prompts used in Stacy's (1997) free association task include homograph words that could have an alcohol or non-alcohol meaning (e.g., draft, pitcher) and expectancy outcomes which may result from a drinking episode (e.g., relaxation, forgetting problems).2 (Although the homograph words used by Stacy are least like the expectancy/attitude words used in the remaining studies, the expectancy outcomes obviously are suited perfectly for this analysis. Complicating their inclusion, though, is that these two item types are grouped together in the Stacy analyses under a factor labeled “Memory Activation.” However, because the outcome expectancy items [which have a value component] load more heavily on Stacy's “Memory Activation” factor than do the homographs, the decision is made to keep this study in the analysis.) Participants are asked to respond with one word to each of these prompts with the “first word it makes you think of.” Responses to each type of prompt are coded in dichotomous fashion; alcohol-related or not. For the explicit task, participants are asked to provide three good or pleasant outcomes of drinking alcohol. They are then asked to rate the likelihood of these outcomes when they drink alcohol.
Results show a significant correlation between the implicit and explicit measures of alcohol expectancy, r (341) = .23. In covariance structure modeling (CSM), both the implicit and explicit measures significantly predict drinking (a composite score including frequency of drinking, frequency of intoxication, and problems resulting from drinking), with the implicit measure accounting for a larger portion of variance (β = .33) than the explicit measure (β = .09).
Unique to this study is some procedural overlap between the implicit and explicit measures. To generate the expectancies for what is considered the explicit measure, participants follow instructions that are consistent with a free association task (i.e., an implicit task; “generate 3 positive consequences of drinking alcohol”). In fact, free association is the very technique used in the implicit portion of the same study. Participants are asked to report the first three “good or pleasant” alcohol effects they could think of. They are then asked to rate the likelihood of each of these effects. Although participants’ ratings of the likelihood of these self-generated outcomes (undertaken subsequent to the free association) legitimately may be considered explicit in nature, these ratings could only be carried out for items that the participant self-generates via free association. The boundary between these tasks (as explicit or implicit) must be considered unclear and, therefore; the effect of such overlap on the operation of a measure (and on the CSM results) is unknown.
As shall be seen subsequently, these results are inconsistent with most other reviewed studies in that the implicit measure out-predicted the explicit measure. Given the sample size (n = 342), it is unlikely that this inconsistency is the result of unreliability of the participant sample. Instead, given that the explicit measure is comprised of only three items, and is by far the briefest of the explicit measures appearing in the reviewed studies, the possibility of unreliability in this explicit measure of alcohol expectancies remains open.
Palfai and Wood (2001) use the outcome expectancy prompts (excluding the homographs) from Stacy's (1997) implicit FA task to make comparisons with a more comprehensive explicit measure of alcohol expectancies, the Kushner, Sher, Wood, and Wood (1994) Expectancy Questionnaire (EQ). The EQ is a Likert-type scale assessing positive and negative alcohol expectancies. Drinking is assessed using questions of frequency, quantity, and heavy drinking. Their intent is to determine if each accounted for a unique portion of variance in drinking even after accounting for the variance in prediction contributed by a presumably more reliable explicit task.
Unlike Stacy (1997), Palfai and Wood (2001) find the explicit measure to account for more of the variance in both drinking frequency and quantity (modeled separately; Betas range from .09 to .47), and to predict drinking independently of the implicit expectancy measure (which also predicts drinking). In addition, significant correlations are found between the implicit and explicit measures of alcohol expectancy, r (313) = .32, and between both types of measure and all drinking indicators (correlations ranged from .15 to .34).
Implicit Association Test (IAT)
Wiers and colleagues introduce the IAT to the alcohol field to assess implicit memory associations between alcohol-related and affective concepts. In Wiers, van Woerden, Smulders, and de Jong (2002), two different versions of the IAT are used to test alcohol associations with affect categorized under the two affective dimensions of valence (e.g., good or bad) and arousal (e.g., energetic, relaxed). In general, the IAT assesses the associations between these affective words and words representing categories such as black people or white people. In this case the categories to be evaluated are alcohol and soda (control). Response keys are assigned to the affective words and are shared with the category words. For example, the left response key may represent “bad” and soda. When presented with either stimulus, the participant is required to respond with a key press. The outcome measure most typically used for the IAT is latency to respond, with the notion being that the degree of association between affect and category yields faster responses.
This study also incorporates two explicit measures of alcohol expectancies: a visual analog scale with items representing valence and arousal dimensions, and an expectancy measure comprised of positive and negative expectancy items. Alcohol use, measured both retrospectively and prospectively, is calculated as the product of frequency and quantity of drinking. The results reveal no significant relationships between either IAT scale and the explicit expectancy measures used. However, both implicit and explicit measures reliably predict retrospective and prospective drinking (βs ranged from −.28 to .54). In both cases, explicitly measured expectancies explain greater drinking variance than expectancies measured using the IAT.
Following Wiers et al. (2002), Jajodia and Earleywine (2003) administer a version of the IAT that addresses the valence dimension of alcohol expectancies. Explicit expectancies are measured using the Alcohol Expectancy Questionnaire (AEQ; Brown, Christiansen, & Goldman, 1987). The AEQ is selected because of its demonstrated predictive and concurrent validity. Alcohol use is measured using indices of quantity and frequency. Neither IAT scale in this study (positive and negative) is significantly correlated with any of the AEQ scales, but the positive IAT and all AEQ scales are significantly correlated with each drinking measure (correlations ranged from .27 to .59). In contrast to the earlier study by Wiers et al. (2002), the negative IAT is not. Consistent with the correlation patterns, three separate regression models show that both the positive IAT and the AEQ independently predict variance in each of the three indices of drinking (Betas ranged from .21 to .43). In each regression model, the explicit measure (AEQ) explains more variance in the drinking outcomes than the implicit measure (IAT).
A second IAT study by Wiers and colleagues (Wiers, van de Luitgaarden, van den Wildenberg, & Smulders 2005) uses essentially the same implicit (with some scoring modifications) and explicit measures of alcohol expectancies as Wiers et al. (2002) to assess change following an expectancy challenge procedure (to preclude confounding the present meta-analysis by inclusion of cognition-drinking relationships following an intervention, only pre-expectancy challenge results are reported here). Both quantity and frequency of drinking are measured. Inconsistent with the earlier Wiers et al. (2002) report, this study reveals a significant correlation between implicitly and explicitly measured arousal (correlations ranged from .19 to .29). Neither the implicit nor the explicit measures are correlated with drinking, however, but instead are significantly related to the extent of alcohol problems. The absence of a relationship between measures of alcohol-related cognition, especially explicit expectancy measures, and measures of drinking is highly unusual within the larger corpus of existing studies of alcohol expectancies and is inconsistent with any of the IAT studies reviewed in this analysis. Pretest regression models predicting drinking are not constructed in this study, so this study is included in the meta-analysis of drinking prediction, but not unique drinking prediction.
Houben and Wiers conduct several studies establishing IAT parameters such as the choice of using a bipolar or unipolar scale (e.g., whether valence should be considered a single dimension of good-bad or two separate dimensions for both good and bad; 2006a), salience asymmetry (whether the categories are primed or not; 2006b), personalization (using alcohol associations provided by the participants; 2007a, 2007b), and contrast categories (the choice of control category; 2008), all with the purpose of predicting drinking. The 2006(b) study also includes another implicit measure, a thought listing task, which is comparable to the FA method used by Stacy (1997) and Palfai and Wood (2001). For each of these studies, they also include explicit measures of expectancies, attitudes, and/or “feelings.” Results across these studies are fairly consistent. Correlations between implicit and explicit measures are in the small to moderate range (.19 to .49). Bivariate (r) prediction of drinking by implicit measures ranges from .31 to .50, with unique prediction (β) ranging from .24 to .36 (regression analyses were not conducted for 2006b). The range for prediction of drinking is a bit larger for the explicit measures with bivariate indices falling between .29 and .58 and unique indices falling between .33 and .51. In two of these studies (2006b; 2007b) the implicit measure has a larger bivariate relationship with drinking than the explicit measure (2006b did not measure unique prediction). All four studies that conduct regression analyses find unique prediction to be larger for explicit measures than implicit measures.
McCarthy and Thompsen (2006) take a psychometric approach to comparing implicit and explicit measures of alcohol expectancies. As in Jajodia and Earleywine (2003), positive and negative alcohol expectancies are assessed separately using the IAT. The AEQ is used as an explicit measure of alcohol expectancies. Both the positive and negative expectancy IATs are re-administered one month later to assess the temporal stability of these measures. Using bivariate correlations and latent variable models, significant relationships are found between IAT and AEQ (.15), between IAT and drinking (rs range from .18 to .21), and between AEQ and drinking (rs range from .24 to .52). Latent variable analysis shows unique prediction of drinking by both IAT (.20) and AEQ (.36). Test-retest coefficients are modest for both the positive (.54) and negative IATs (.44).
Because McCarthy and Thompsen (2006) and Jajodia and Earleywine (2003) use the same implicit and explicit tasks, and similar items in the IAT (alcohol expectancy words), it would be anticipated that the effect sizes of these studies would be similar. Encouragingly, effect sizes in each of these studies are fairly consistent. The main discrepancy is a significant relationship between implicit and explicit measures that appears in McCarthy and Thompsen (2006), but not in Jajodia and Earleywine (2003). Because the effect size is larger in the latter study (.17 vs. .15), it is likely that the significant relationship between the implicit and explicit measures in McCarthy and Thompsen (2006) results from the larger sample size.
Extrinsic Affective Simon Task
Conceptually similar to the IAT, the EAST requires participants to pair task-relevant cues (in this case alcohol words) with affective categories, and then measures the latency to respond to cues on a computer keyboard. The EAST was designed to overcome some of the weaknesses of the IAT, primarily the pitting of one type of stimulus (alcohol) with another (soda) to establish the valence of memory associations (De Houwer, 2003). De Houwer & De Bruycker (2007) administer such a version of the EAST task along with an explicit measure of attitudes toward beer. Results show the strongest relation of any of the studies between the implicit and explicit measure (r = .48). Significant relationships are also found between the EAST and drinking frequency (rs < .51) and between the explicit attitudes measure and drinking frequency (rs < .75). Similar to most of the IAT studies, the explicit attitudes measure explains more unique drinking variance (Betas < .47) than the EAST (Betas < .33). Although explaining less unique variance than the explicit measure, the EAST in this study had the largest unique effect size (equal to Stacy, 1997) of any of the implicit measures.
Also using the EAST as an implicit measure of alcohol attitude associations and an explicit measure of attitudes toward alcohol, de Jong et al. (2007) find similar results and comparable effect sizes to De Houwer and De Bruycker (2007). The major distinction is an inability to detect a significant relationship between the implicit and explicit measure in this study. This may be a function of this study's small n (32), but nonetheless the observed effect was small (r = .19).
Other implicit tasks
Kramer and Goldman (2003) use a modified Stroop task, which, like the IAT and EAST, measures reaction time to expectancy-consistent stimuli following an alcohol prime. Primes consist of either alcohol or alcohol-neutral beverage words, and participants name the ink color in which alcohol expectancy target words are printed. Target expectancies are selected to represent valence and arousal dimensions. The Alcohol Expectancy Multiaxial Inventory (AEMax; Goldman & Darkes, 2004) is the explicit measure in this study. Results reflect strong associations between the alcohol primes and sedating expectancy words for light drinkers relative to heavy drinkers, and strong associations between alcohol and arousing expectancies for heavier drinkers relative to light drinkers. Both Stroop (implicit) and AEMax (explicit) scores are significantly correlated with both typical frequency and quantity of alcohol consumption (correlations range from .23 to .50). These measures are not correlated with one another, however, and, as might therefore be anticipated, simultaneous multiple regression predicting a quantity and frequency drinking composite show independent prediction (Betas range from .24 to .52).
Taking a two-in-one approach, Read, Wood, Lejuez, Palfai, & Slack (2004) “piggyback” an index usually taken to reference implicit processes upon an explicit self-report alcohol expectancy measure. Assessing what they term “expectancy accessibility,” latency (reaction time) to endorse expectancy items is recorded. Drinking in this study is measured only as quantity (typical number of drinks per occasion). Results show similar relationships between the implicit and explicit scores, implicit scores and drinking, and explicit scores and drinking; (r = .36, .36, .37, p < .01, respectively). Although regression models are provided for this study, none include both the implicit and explicit indices as predictors of drinking. As a result, this study is not included in the comparisons of unique prediction.
The Deese-Roediger-McDermott (DRM) false memory paradigm assesses the degree to which individuals falsely recall having studied words semantically related to a study list. Higher rates of false memory are thought to indicate greater associations between studied words and denser networks of such associations (McEvoy, Nelson, & Komatsu, 1999). Reich, Goldman, & Noll (2004) adapt this task to assess alcohol expectancy memory associations by having participants study alcohol expectancy word lists in a simulated bar. The false memory list created for this study includes 12 presented alcohol expectancy words and 3 non-presented targets. Like the items in the other implicit tasks, these items fit into the affective dimensions of valence and arousal. The DRM false memory measure and several subscales of the AEMax (a 24-item version), the explicit alcohol expectancy measure included in this study, are significantly correlated with typical quantity of alcohol consumption per drinking occasion (correlations ranged from −.20 to .40). A small but significant correlation is also found between false memory scores and the positive subscale of the AEMax, r (162) = .18. When all measures are entered into a simultaneous regression (the false memory score and the eight subscale scores of the AEMax), false memory scores and the positive AEMax subscale emerge as significant independent predictors of drinking quantity (β = .16 and .26, respectively).
Combining Implicit/Explicit Effect Sizes Using Meta-Analysis
Twelve out of sixteen studies report a significant correlation between implicit and explicit measures of alcohol expectancy. The mean weighted effect across all studies is .25 (.12), demonstrating small but significant measurement overlap. All sixteen studies report significant prediction of drinking (or drinking problems) by implicit cognitive measures with a mean weighted effect of .35 (.15), while 13 report significant unique prediction of drinking, with a mean weighted effect of .23 (.08) (the only studies that do not report unique prediction do not test for it). The .35 mean effect size for implicit prediction of drinking is larger than the .23 observed by Rooke et al. (2008) and likely the result of our choice of the largest effect size to represent each study. With respect to explicit alcohol expectancy measures and drinking (or drinking problems), all studies report significant correlations with a mean weighted effect size of .41 (.17), and 13 report significant unique prediction with a mean weighted effect of .29 (.15). Figure 1 depicts the combined r values, while Figure 2 depicts the combined βs.
Comparing Implicit and Explicit Effects
Weighted mean implicit effect sizes are compared with explicit effect sizes using the Wilcoxon Signed Ranks Test. Thirteen out of sixteen studies show reliably larger effects for explicit measures than implicit measures, Z(15) = 2.12 (p = .034). Unique prediction of drinking also significantly differs by type of task, Z(12) = 2.27 (p = .023).
Moderator Analysis
To rule out the possibility that the results would vary based on the drinking outcome used, we conduct two moderator analyses, one for drinking quantity and the other for drinking frequency, including only studies in which these effects are presented in the published manuscript. Seven of the sixteen studies measure bivariate correlations3 between implicit and explicit measures and drinking quantity and (6 out of 16 for drinking frequency). These seven studies represent all of the different implicit tasks included in the full meta-analysis, which supports the notion that this sub-sample is representative. It should be noted, however, that the report by Stacy (1997) did not include these effects and is not represented in the moderator analysis. This particular study is noted because its results varied most from the general pattern of the other studies. In all studies of this sub-sample, explicit effects are larger than implicit effects for both quantity and frequency. For drinking quantity and implicit and explicit tasks, the mean weighted correlations are .21(.12) and .40(.24), respectively. For drinking frequency the mean weighted correlations are .22(.16) and .46(.21), respectively. Even with the smaller sub-sample, these mean effects differ significantly from each other for quantity, Z(6) = 2.37, p = .018, and for frequency, Z(5) = 2.20, p = .028. The results of this analysis make it unlikely that the results of the full meta-analysis are moderated by the drinking outcome chosen.
Discussion
This meta-analytic review is designed to follow up on fundamental questions about the nature of explicit and implicit assessment as it has been extended into alcohol expectancy and other alcohol-related cognition research. Two primary questions are addressed: First, do implicit measures assess a distinct aspect of the alcohol cognition domain not accessible via explicit measurement, or, alternatively, do they just offer a new set of indices of the same alcohol-related cognitive constructs defined previously by explicit measures? Second, at a practical level, could prediction of drinking using alcohol-related cognitive constructs be improved by the addition of implicit measures (that is, do implicit expectancy measures add unique predictive power in the prediction of drinking beyond that contributed by explicit measures)? Obviously these questions are linked—if overlap between the implicit and explicit measures approaches 100% (that is, they essentially assess the same construct), then implicit measures could not offer incremental prediction. As with most psychological measures, however, the results of our meta-analysis fall somewhere in between; implicit and explicit measures overlap considerably (they do not probe completely distinct aspects of the cognitive domain), but they do not completely account for the same predictive variance in drinking, revealing some psychometric separation.
In eleven out of sixteen studies reviewed, implicit and explicit measures are significantly correlated with one another, albeit with relatively small effect sizes (mean r = .25). More to the point, in thirteen of the sixteen studies, explicit measures account for more drinking variance than implicit measures (in 12 out of 13 studies for unique variance). This qualitative difference is supported by the significant differences observed in the statistical comparison of mean correlation values and Beta values (unique variance explained). It is also noteworthy, though, that even with the relatively large amount of variance in drinking explained by explicit measures overall, implicit measures do account for significant additional drinking variance.
Despite the relatively large amounts of drinking accounted for by the measures cumulatively, the effect sizes remain modest, leaving large amounts of drinking variance unaccounted. The range of effects can be seen in Table 1, and is underscored by the relatively large error bars observed in Figures 1 and and2.2. This range reflects what can be observed in this body of studies; that even with the same measure (e.g., IAT), the same participant population, and even the same researchers, effect sizes vary (see the Houben & Wiers studies where IAT-drinking correlations ranged from .29 to .50). Despite this within-group variation, however, the relative effects between implicit and explicit measures are maintained across studies.
The consistency across individual studies in this meta-analysis is remarkable in light of the heterogeneity of both the implicit and explicit measures used. Scores derived from the implicit measures are especially diverse, and range from the number of free associate responses, reaction times in milliseconds, and the number of falsely recognized words. Finding consistent results given the variations in procedure suggests both that this meta-analysis is generalizable and that these findings can usefully inform future work in this domain; i.e., in the absence of novel approaches to implicit measurement, the present pattern of results appears unlikely to change.
In addition, the consistency of the obtained relationship between implicit and explicit measures across what are typically considered different types of cognition (in this case expectancies, attitudes, and other cognitive [associational] processes) supports their inclusion in the present meta-analysis. It must be kept in mind, however, that although these different cognitive domains may have different theoretical components, the studies included in this meta-analysis were selected based on similarity of the operational definitions measuring each cognitive domain (and not on the full set of operations that may be used to probe these processes). That is, the operational measures of these domains included herein were constrained by the need for combining them into a meta-analysis. Because many other operations may be used to define each of these domains (e.g., expectancy defined as dopamine release in the nucleus accumbens), the present findings are uninformative as to the extended relative merit of these theoretical approaches. Instead, these theoretical frameworks must be judged on the extent and quality of their full nomothetic network of explained findings. We also suspect, however, that these theoretical frameworks ultimately only represent differing areas of emphasis (and disciplinary traditions) in what will eventually be seen as integrated complex systems of information processing and decision-making. For example, attitude research emphasizes the affect-cognition linkage, and expectancy emphasizes the anticipatory nature of information processing; both characteristics are, of course, essential to real world survival.
Other studies using only implicit measures in the alcohol domain also support the inclusion of both implicit and explicit measures in alcohol research. These studies have used implicit primes (exposure to performance-affecting stimuli without participants’ conscious notice of that stimuli or its intended influence) to influence tasks that have traditionally been considered explicit (e.g., recall; Reich, Noll & Goldman, 2005; an alcohol expectancy paper and pencil measure; Wall, Hinson, McKee, & Goldstein, 2001). Additionally, and providing the strongest rationale for the continued study of implicit memory effects on drinking, (implicit) primes have been shown to impact quantity of consumption in studies of ad lib drinking (Roehrich & Goldman, 1995; Carter, McNair, Corbin, & Black, 1998; Stein, Goldman, & Del Boca, 2000). Conceptual Questions
Implications For Elucidating Memory Process(es)
Because the results of this meta-analysis indicate that there is both overlap and uniqueness in the capacity of implicit and explicit tasks to predict drinking outcomes, they are agnostic as to the fundamental issues noted earlier about the nature of underlying memory process(es). They could have been obtained from one memory system that is responsible for the overlap and the uniqueness with specific task components tapping unique kinds of information from this one system; from two systems that operate separately but show interplay; or from an unspecified number of memory components/systems that network to control outputs. In keeping with Frank, Cohen and Sanfy (2009, p. 73) who recently describe the two-system approach as “both too simplistic and too vague,” we lean toward and have argued elsewhere on behalf of the third option (Goldman, Reich, & Darkes, 2006; Goldman, Darkes, Reich, & Brandon, 2006; see also Roediger, 2003 and Roediger & Amir, 2005, for an earlier presentation of this position). (If, of course, we refer to two somewhat overlapping categories of processes that each includes multiple memory components, both views can be accommodated.) At purely the measurement level, however, these results support a model in which prediction of drinking might be optimized by combining the best assortment of implicit/explicit tasks. In this combination, it must be remembered that the extent to which explicit and implicit measures overlap within their own domains remains unspecified at this point.
Psychometrics
The difference in predictive variance accounted for between explicit and implicit measures may be due to measurement error (unreliability) in the tasks, systematic differences between the tasks (method variance), or a small but real distinction between aspects of the constructs measured by the tasks (validity). Because the current pattern of effect sizes mimics those in other areas of inquiry (low correlations with explicit measures in social psychology and smoking research; Fazio and Olson, 2003, Waters and Sayette, 2006), there is reason to be attentive to the reliability of each implicit instrument. Testing the reliability of implicit tasks in general has yielded questionable results, especially regarding temporal stability (test-retest reliability). These results are most evident in priming tasks where effects are not greater than .30 (Cunningham, Preacher, & Banaji, 2001). The IAT seems most resilient to instability with short-term (1-month) consistency being modest (.62) and with the expected decreases in time (.48 at 1-year) (Egloff, Schwerdtfeger, & Schmukle, 2005). Either way, these values are much lower than the conventional standards of traditional psychological measures. Even though two studies report acceptable internal consistency of a given implicit alcohol expectancy measure (Wiers et al., 2002; Wiers et al., 2005), and a few studies have shown modest test-retest reliabilities (Wiers et al., 2005; Reich, Brandon, Morean, & Goldman, 2005; McCarthy & Thompsen, 2006), sufficient error can be observed in all the implicit measures to at least partially explain the low correlations observed. The possibility also remains that error resulting from demand characteristics is inflating the relationship between explicit measures and drinking. A psychometric strength of implicit instruments is that they are less susceptible, at least theoretically, to such demand characteristics.
The possible influence of method differences (i.e., method variance) is impossible to determine using the studies included in this review because different constructs (separate from alcohol-related cognition) are not evaluated using the same task. Because task-specific factors such as instructional set have previously been demonstrated to affect relationships between implicit and explicit tasks (Olson & Fazio, 2003; Nosek, 2007), method differences remain a possible source of extra-construct error in these studies—especially because implicit tasks vary in terms of not only instructional set, but response type (e.g., verbal vs. reaction time). Given that in this meta-analysis we are not able to control for these sources of variation, it is difficult to conclusively say whether the overlap observed reflects the full extent of overlap in the underlying construct (that is, the extent to which explicit and implicit measures of alcohol expectancy measure the same thing) or overlap in the method.
The sparseness of research targeted at the psychometric evaluation of implicit measures is not due to negligence. Indeed, the obstacles to within-subjects comparison of implicit tasks are formidable. For example, priming effects can be vastly different with only slight changes in stimuli; Reich, Noll, & Goldman (2005) demonstrate the sensitivity to slight modifications in alcohol expectancy task stimuli by simply varying the first word of a list to be later recalled (beer versus milk). Participants in a priming study therefore must remain naïve to other relevant stimuli to avoid task contamination. As a consequence, pure test-retest evaluations are virtually impossible because it is difficult to avoid having the first administration contaminate the second. Olson and Fazio (2003) also showed that simply changing the instructional set of the same implicit task led to significant differences in scores. The instructional set of a second administration of an implicit task is likely experienced differently because it is influenced by the experience gained during the first administration.
This response sensitivity to very subtle contextual differences also suggests that low reliability of implicit measures might be a consequence of their state-like nature (see Gawronski, LeBel, & Peters, 2007). This notion is underscored by the decreased temporal stability of implicit measures mentioned above (Cunningham et al. 2001; Egloff et al. 2005). Consistent with the previously mentioned expectancy studies of the neurophysiology of decision making studies (e.g., Campos et al., 2005; Kobayashi et al., 2002), the perception of motion (Kerzel, 2005), and the perception of time (Correa et al., 2005), perhaps true implicit processes are “meant” to be sensitive to short, or medium term variations in context. Measured across wider time frames or varying contexts, therefore, they may appear less reliable than explicit measures because they accurately index fluctuating neuropsychological processes (as do Event Related Potentials or fMRIs; see also De Houwer, 2006). Cognitive scientists have acknowledged the substantial influence of context. For example, Nelson, McEvoy, & Pointer (2003) applied a model originally used in quantum physics, in which associative patterns exist in a state of “superposition” (undifferentiated), and are only “collapsed” into their functional state upon exposure to a particular context.
File-drawer issues
As with all meta-analyses, identifying all research relevant to this project is difficult and likely biased toward studies with positive outcomes or larger effect sizes (Rosenthal & DiMatteo, 2001). The question arises as to whether the results of this meta-analysis would hold with the inclusion of unpublished studies in this arena. Two lines of evidence support that they would: First, the consistency of effects observed across the sixteen diverse studies with 1,857 participants makes it unlikely that these studies were found by chance. Second, the moderator analysis demonstrates that the pattern of results is consistent across multiple measures of drinking. These observations underscore the confidence that can be placed in the meta-analytic effects observed here.
Conclusion
Finally, the body of research summarized here both qualitatively and quantitatively supports the use of implicit measures of alcohol expectancies, attitudes, or implicit cognitions in the prediction of alcohol use. Although direct comparisons with explicit measures indicate the advantage of explicit measures, the added value offered by implicit measurement is supported. The extent to which neurophysiological “memory systems” underlying these effects overlap cannot be determined using currently available studies (and perhaps methods); further theoretical and empirical investigation is necessary. Despite this constraint on our capacity to understand the operation of memory, the expanding use of implicit measures clearly has advanced our understanding of cognitive/affective/information processing as an influence on alcohol use.
Acknowledgments
This study was supported in part by National Institute on Alcohol and Alcoholism Grant R01 AA08333. We thank Howard Steinberg, Fran Del Boca, Jack Darkes, Paul Greenbaum, and Karen Obremski Brandon for their comments on earlier drafts of this manuscript.
Footnotes
Publisher's Disclaimer: The following manuscript is the final accepted manuscript. It has not been subjected to the final copyediting, fact-checking, and proofreading required for formal publication. It is not the definitive, publisher-authenticated version. The American Psychological Association and its Council of Editors disclaim any responsibility or liabilities for errors or omissions of this manuscript version, any version derived from this manuscript by NIH, or other third parties. The published version is available at www.apa.org/journals/adb
1Because expectancy measures tend to include both positive and negative items, correlations with drinking outcomes in these studies are both positive and negative. So that the strength of relationship could be assessed independent of the direction of the relationship, the absolute value of effect sizes is used.
2Stacy (1997) also uses an object association task, but scores on this task were dropped from further analyses due to low correlation with the other free associate prompts.
3An insufficient number of studies examined unique prediction of drinking quantity and frequency to conduct meaningful moderator analyses.
References
*References marked with an asterisk indicate studies included in the meta-analysis.
