Task-Dependent Electrophysiological Signatures of Explicit Perceptual Timing

Studies investigating the neural mechanisms of time perception often measure brain activity while participants perform a temporal task. However, several of these studies are based exclusively on tasks in which time is relevant, making it hard to dissociate activity related to temporal processing from other types of temporally structured brain patterns. In the present study, human participants performed a temporal or color discrimination task of visual stimuli. In different blocks, participants were informed which magnitude they would have to judge before or after presenting the two stimuli (S1 and S2). Our behavioral results showed, as expected, that performance was better when participants knew beforehand which magnitude they would judge. Electrophysiological data (EEG) was analyzed using Linear Discriminant Contrasts (LDC) and a Representational Similarity Analysis (RSA) approach to investigate whether and when information about time and color was encoded. During the presentation of S1, we did not find consistent differences in EEG activity as a function of the task. On the other hand, during S2, we found that temporal and color information was encoded in a task-relevant manner. Taken together, our results suggest that task goals strongly modulate the encoding of temporal information in EEG activity.


Introduction
Perceptual timing is essential for humans and other animals to interact with their environments. Several studies combined temporal tasks with concurrent neuroimaging methods, such as M/EEG, to investigate the underlying brain mechanisms of temporal processing. A commonly used task in these studies is temporal discrimination, in which participants have to judge whether a given duration is shorter or longer than a reference. Different M/EEG markers have been proposed to be correlated with temporal processing based on this task, such as early post-interval N1P2 component (Kononowicz & van Rijn, 2014) and the late positive component of timing (LPCt; Bannier, Wearden, Le Dantec & Rebaï, 2019;Wiener & Thompson, 2015;Paul et al., 2003). However, the majority of these studies rely solely on temporal tasks. For this reason, it is hard to dissociate what aspects of EEG activity are associated with temporal processing or with general task-related decisions.
To address this limitation, some investigations have compared temporal discrimination tasks with discrimination of other attributes, such as color (Coull, Vidal, Nazarian & Macar, 2004;Kulashekhar, Pekkola, Palva & Palva, 2016), size (Gontier, Dantec, Paul, Bernard, Lalonde & Rebai, 2008) and space (Coull, Charras, Donadieu, Droit-Volet, & Vidal, 2015). For example, Coull and colleagues (2004), using functional Magnetic Resonance Imaging, found a higher activation of areas such as the pre-SMA and a network of other cortical and striatal areas when participants paid more attention to the duration than the color of a stimulus. Recently, Kulashekhar and colleagues (2016) used a similar design combined with MEG to investigate the possible role of oscillations in temporal processing. In their findings, beta oscillations were correlated to time estimation compared to the otherwise similar color information estimation.
Here, we aimed to examine human electroencephalogram (EEG) using a duration and color discrimination task. As in the study of Coull and colleagues, we controlled how much attention was allocated to different dimensions (time or color) by informing participants whether they would make a judgment about time or color before the stimuli (Pure Blocks) or only after the end of the trial (Mixed Blocks). Contrary to previous experiments, we parametrically varied the difference in time and color between stimuli, allowing an in-depth investigation of whether information about time or color in EEG activity was task-dependent. We compared distances between patterns of EEG activity and investigated how these distances were modulated by task, by time or by color and whether possible modulations depended on the task to be performed. We found a weak pattern of differences when participants were exposed to the duration or color to be stored for further comparison. However, during the presentation of the comparison event, there were clear task-dependant differences in EEG activity. Importantly, information about time modulated EEG activity more strongly when it was relevant and correlated with behavioral performance.

Participants
Twenty-one human volunteers (age range, 21-32 years; 11 females) participated in the experiment. All of them had a normal or corrected-to-normal vision and did not report any psychological or neurological diagnoses. The Research Ethics Committee of the Federal University of ABC approved the experimental protocol (CAEE: 38370314.0.0000.5594), and the experiment was performed following the approved guidelines and regulations. Data from one volunteer (age 24, female, not included in the twenty-one participants above) were excluded from the analyses due to excessive noise and artifacts in the EEG signal (proportion of rejected trials above 25%).

Stimuli and Procedures
The experiment consisted of a time or color discrimination task (figure 1). The stimuli were presented using Psychtoolbox (Brainard, 1997) v.3.0 package for MATLAB on a 17-inch CRT monitor with a vertical refresh rate of 60 Hz, placed approximately in a viewing distance of 50 cm from the participant. Responses were collected via a response box of 9 buttons (DirectIN High-Speed Button; Empirisoft). We used the left and right buttons for responses in which participants should respond using both hands. We presented 720 trials consisting of two visual stimuli (filled circles) with different colors and durations. Participants were instructed to answer if the second stimulus was shorter/longer in duration or redder/bluer than the first one. The magnitude to judge was determined by the block condition: (1) In Time Pure and Color Pure blocks (2 blocks of each), participants were informed beforehand whether to judge differences in duration or color between the two visual stimuli; (2) In Mixed blocks (the remaining four blocks), participants would only know the magnitude (time or color) to judge during the response screen, 500ms after the offset of the second stimulus. Block order was randomized for all participants, and the background color was gray (RGB-color 100; 100; 100).
Each trial started with the presentation of a circle (S1, one visual degree radius) at the center of the screen with a duration randomly chosen between 750 ms to 1500 ms, and colored in the RGB space [1-C, 0, C], in which C could range randomly from 0.2 to 0.5. The RGB space and parameter C for manipulating color was chosen based on de Gardelle and Summerfield (2011). After a random ISI of 400 ms to 600 ms, in which only a fixation point was present (0.25 visual degree radius), a second circle (S2, one visual degree radius) appeared with a different duration and color. Duration and color (controlled by parameter C) of S2 could be a percentage of duration and C of the first stimulus, ranging from -80% to 80% (6 possibilities in total: -80%, -40%, -20%, +20%, +40%, +80%) Durations and colors were independently randomized, and thus, orthogonal. After a delay of 500 ms, a response screen was presented in which participants were instructed to judge the duration or color of S2 relative to S1. In Pure blocks, the response screen reminded participants which dimension to be compared, while in Mixed blocks, the response screen informed which dimension should be compared.

EEG recordings and pre-processing
EEG was recorded continuously from 64 ActiCap Electrodes (Brain Products) at 1000 Hz by a QuickAmp amplifier (Brain Products). All sites were referenced to FCz and grounded to AFz. The electrodes were positioned according to the International 10-10 system. Additional bipolar electrodes registered the electrooculogram (EOG). Data pre-processing was carried out using FieldTrip (Oostenveld, Fries, Maris & Schoffelen, 2011) toolbox for MATLAB. We segmented the data in four different epochs, for the period during the presentation of S1 and S2 (S1/S2 onset analysis) and just after the offset of each stimulus (S1/S2 offset analysis). Filters were applied to the continuous data with a bandpass of 0.05 Hz to 30 Hz for the S1/S2 onset and 0.1 Hz to 30 Hz for S1/S2 offset (Butterworth filter, order 3). The higher initial cut-off for the filter in the S1/S2 offset analysis was done to avoid slow fluctuations during the stimulus presentation. All data were downsampled to 256Hz and re-referenced to the activity of electrodes TP9 and TP10, located in the earlobes.
For the S1/S2 onset analysis, epochs were locked at the onset of S1/S2, and data were segmented from -150 ms to 750 ms (748.4ms due to downsampling). For the S1/S2 offset analysis, epochs were locked at the offset of S1/S2, and data were segmented from -150 ms to 400 ms (396.9 ms) for S1, and to 500 ms (498.4 ms) for S2. Channels with missing data due to problems in acquisition or channels with excessive noise were interpolated with neighbor channels using FieldTrip channel repair function. Data from most participants had none or up to two channels interpolated. Only two participants had 3 and 4 channels interpolated.
For eye movement artifact rejection, an independent component analysis (ICA) was performed for S1 onset segments. Eye-related components were identified by visual inspection of topographies and time series from each component. Eye related components were rejected for all segments. Baseline correction was performed using the periods from 150 ms before S1/S2 onset and 50 ms before and 50 ms after S1/S2 offset. We rejected trials separately for the onset and offset periods due to the segments' different filtering and duration. We used FieldTrip functions to visually identify and exclude trials in which the absolute maximum potential exceeded 200 μV for S1/S2 onset and 150 μV for S1/S2 offset. The remaining trials were rejected for noise by visual inspection. The percentage of rejected trials for S1 onset segment was 3.23% (range between 0% -11.11%), for S2 onset was 3.53% (0% -12.64%), for S1 offset was 2.14% (0.14%-7.50%) and for S2 offset was 1.50% (0.14% -5.55%).

Behavioral Analysis
Behavioral analysis was based on the proportions of each type of response (longer/shorter or redder/bluer) as a function of the time or proportional color of the second stimulus (S2) relative to the first stimulus (S1). We estimated psychometric functions for each participant in different conditions: Time Pure, Color Pure, and for the mixed blocks, we separated the data in trials in which participants were asked about duration (Time Mixed) and color (Color Mixed).
We fitted cumulative normal psychometric functions for each participant and condition, defined by four parameters: threshold, slope, lapse-rate, and guess-rate (Wichmann & Hill, 2001). Guess rates and lapse rates were restricted to a maximum of 0.05. Each function's four parameters were fitted using maximum likelihood estimation as implemented in the Palamedes Toolbox (Prins & Kingdom, 2018). To evaluate participants' performance, we estimated the Point of Subjective Equality (PSE) and the JND (Just Noticeable Difference). The JND is defined as the difference from 25% to 75% estimates of the psychometric curve, divided by two. This measurement represents how much different one stimulus has to be relative to another so that participants can notice. In contrast, the PSE represents the magnitude difference by which the second stimulus is equally likely to be judged as longer/shorter or redder/bluer than that of a first stimulus. We compared the JND and PSE from the Pure Blocks to their counterparts in the Mixed Blocks using a paired t-test. Effect sizes were estimated using Cohen's d as implemented in JASP (2020).

Multivariate Pattern Analysis
To compare the pattern of EEG activity across different conditions, we used Linear Discriminant Contrasts (LDC, Walther et al., 2016). We used this method to estimate distances in the time-resolved EEG signals from different tasks and conditions for each participant. All EEG electrodes were used in this analysis, except for the reference ones (TP9 and TP10). The LDC is a cross-validated Mahalanobis distance and allows the interpretation of ratios between distances, as its null distance is zero (Walther et al., 2016). The LDC is calculated as: matrices were then entered into a simple linear regression analysis, separately by condition and segments of the experiment (S1 or S2, onset or offset) for each time point. The data-derived distances were dependent variables, and the theoretical distances matrices the independent variables. The estimated coefficients were compared to zero using a similar mass univariate approach as described above.

Behavioral Results
The behavioral results (figure 1) showed that sensitivity, measured by the JND (Just

Exposure phase: No consistent differences in EEG activity by task-goals during S1
Task-related activity: In a first analysis, we focused on how activity evoked by S1 was modulated by the task to be executed. We aimed to investigate whether paying attention to time iii) between mixed conditions. The LDC was estimated from 150 ms before to 748.4 ms after S1 onset (given that the shortest possible duration of S1 was of 750 ms, this means that all trials were used). For S1 offset, we evaluated the EEG signal from 150 ms before S1 offset up to 396.9 ms (given that the shortest interval between S1 and S2 was 400 ms).
The mean distances between tasks during the exposure phase (S1) are shown in figure   2A, and the spatial-temporal distribution of Event-Related Potentials (ERPs) for each task and period are shown in figure 2B. Although there were short periods in which LDC exceeded the critical t-value (3.4613) in Color Pure vs. Color Mixed conditions, distances between tasks were in general small and not significant. A similar pattern was observed for S1 offset, in which there were stronger distances between pure tasks, although not significant.
Time and color-related activity: In a second analysis, we investigated if the duration or color of S1 modulated EEG activity. We aimed to test whether stimuli of different colors or durations evoked different patterns of EEG activity and whether this difference was more robust when that specific dimension was task-relevant. Color information was evaluated in the time-resolved S1 onset signal up to 748.4 ms. Given that information on how much time has passed since S1 onset was only available at S1 offset, we evaluated time information only at S1 offset. For each participant, S1 duration (from 750 ms to 1500 ms) or its color (indexed by the C parameter) were binned into six equally sized bins. The mean duration or mean C for each bin was used to calculate pairwise distances and build theoretical matrices. Pairwise LDC was calculated for each comparison. As explained in the methods section, these resulting pairwise LDC and the theoretical distances' matrices were then entered into a linear regression analysis.
In general, there were no consistent modulations of the EEG signal by color. For the duration, we found two small periods in which coefficients were larger than zero:

Decisional phase: Consistent differences between tasks during S2
Task-related activity: In a next step, we focused on activity evoked by S2, the comparison stimulus. We performed the same LDC analysis to compare tasks during the second stimuli (S2 onset segments). To have a good number of trials of each condition and to have a considerable amount of time points during S2, this analysis was performed on data from trials in which the second stimulus lasted at least 750 ms.
For S2 onset, there was a consistent difference in the EEG signal between pure tasks (green line in figure 3A left, window tested = -150 ms to 748.4 ms; critical t = 3.3991, significant distances from 225.0 ms to 748.4 ms). We also found differences for Time Pure and Time Mixed (purple line in figure 3A left, critical t = 3.3669, from 459.4 ms to 478.9 ms, from 568.8 ms to 572.7 ms and from 615.6 ms to 623.4 ms) and Color Pure to Color Mixed (pink line in figure   3A left, critical t = 3.4030, from 295.3 ms to 389.1 ms, from 420.3 ms to 568.8 ms, from 576.6 ms to 748.4 ms). The spatial-temporal ERPs illustrate these differences measured by LDC from these conditions (figure 3B, left). As expected, no significant distance was found between mixed tasks. In all significant comparisons, the difference was strongly driven by a centro-parietal p300 like response present in trials in which participants have to decide on the color of S2 (mixed blocks and pure color blocks).
The same analysis was conducted for S2 offset, from 150 ms before the offset of the stimulus to 498.4 ms after. Trials in which the second stimuli lasted less than 300 ms were excluded from this analysis to reduce sensory ERPs' contamination. Again, there was a significant distance between pure tasks (green line in figure 3A  and pure time blocks). However, contrary to the p300 like activity, these differences seem to be more concentrated in frontal-central sensors.

Task-relevant features modulate post-interval EEG activity
Decision-related activity: We examined the modulation of EEG activity as a function of the stimulus magnitude of S2 relative to S1. An RSA approach was used to compare activity evoked by stimuli representing different proportions in time (proportional time) or color (proportional color) from S2 to S1, condition-wise. For time information, this analysis was performed on EEG activity of S2 offset (from 150 ms before the offset of the stimulus to 498.4 ms after) since full temporal information would be available only when the interval had elapsed. For color, this analysis was done for S2 onset up to 748.4 ms (for trials longer than this duration at S2), and we used the absolute color proportion from S2 to S1. We calculated pairwise LDCs of the EEG signal for all possible time proportions of S2 relative to S1 (-40%, -20%, +20%, +40%, +80) and for the absolute color proportion (20%, 40% and 80%). Importantly, for the time analysis, the -80% (in duration) was not included, given that this condition had only very short durations (maximum of 300 ms) to avoid false positives due to remaining evoked potentials from the onset and offset of the short duration visual stimulus. Also, for the color analysis, only trials in which the second stimulus lasted at least 300ms were included. These proportions were used to calculate pairwise distances and build two theoretical matrices, one relative to distances in time and one relative to distances in color (figures 4A). As before, these matrices were entered into simple linear regression analyses, separately by condition, and coefficient estimates were evaluated.
As can be seen in Figure 4 (right column), we did not find a significant relation between for Color Mixed (window tested = -150 ms to 498.4 ms; critical t = 3.3604, at time point -32.8 ms (before S2 offset) and from 361.7 ms to 365.6 ms).
The modulation of EEG activity by proportional time information can be seen in figure   4B. Based on the RSA analysis and the spatial-temporal evoked activity of S2 offset (

The relation between post-interval activity, time and behavior is task-dependent
To evaluate whether there is a relation between the post-interval activity and the response given by participants, we performed a binomial regression in which the binary answer (shorter or longer) was used as the response variable and the proportional time (5 values, 0.6, 0.8, 1.2, 1.4 and 1.8) and EEG activity residuals were explanatory variables. We performed this regression for all conditions for the two different sets of electrodes and the two different windows after S2 offset (the same windows and electrodes explored at EEG activity topographies in figure 4B). We calculated EEG activity residuals by subtracting the mean EEG amplitude for that specific proportion of S2 and condition in each electrode and time point. The residuals of EEG activity were used to minimize the correlation between proportional time and EEG activity. This allowed us to investigate whether trial-by-trial EEG fluctuations covaried with behavior.
As expected, proportional time was a significant predictor of behavior for all

Discussion
In the present study, we investigated the neural correlates of temporal processing. To dissociate time-related EEG from other task-related decisions, we used multivariate analyses to compare activity across two discrimination tasks: one in which participants had to compare the color of two stimuli and one they had to compare their durations. Across different blocks, participants did or did not have prior knowledge about what feature would have to be compared.
As expected, our behavioral results showed a better performance in conditions that participants knew the feature to be compared.
Our task consisted of the encoding of duration/color information of a first stimulus and a decision about these features in a second stimulus. We were able to compare whether and how EEG activity differed due to task-goals in these two phases. In general, we observed that: (1) During the encoding phase, EEG activity did not differ strongly across tasks; (2) During the decision phase, there were substantial differences in EEG activity across tasks and how task-relevant features modulated this activity.
Our task design allowed a temporal separation of when information about each feature was accessible to participants during the decision phase. While color information was available at the onset of S2, information about time was fully present only at its offset. This separation was reflected in the EEG signal, where making decisions about color or time evoked activity at the onset and offset of S2, respectively. When making color decisions, there was a clear p300 at S2 onset, in agreement with proposals of this potential reflecting decision-making processes (O'connell et al., 2012;Twomey et al., 2015). Although the p300 was not strongly modulated by how different S2 was relative to S1, this could be due to the color task being slightly easier for participants. On the other hand, decisions about time evoked a more robust pattern of activity in fronto-central sensors, only at the offset of S2. Critically, this offset activity modulation by duration was most reliable when the time was task-relevant, and weaker when it was irrelevant or only possibly relevant.
Our findings are in partial agreement with a recent study (Kulashekhar et al., 2016) that did not find differences in the time-locked signal during the encoding period in MEG recordings.
However, contrary to our findings, the authors did not find differences in the time-locked signal during the decision period. There were substantial differences between our tasks that might explain the contrast between these findings. In their work, the color task consisted of a varying hue that changed during the whole trial from a bluish to a reddish-purple. This was done as an attempt to make the color and the temporal task more comparable, given that participants would have to integrate information across the whole trial to make their color decision in a way similar to the temporal task (Coull et al., 2004 andKulashekhar et al., 2016). On our task, on the other hand, the color decision could be made at the onset of the second event. For this reason, both tasks bring essential and complementary information on how temporal processing is accomplished.
An increasing number of studies have suggested that EEG markers at the end of the interval are correlated with temporal processing, such as early post-interval N1P2 component (Kononowicz & van Rijn, 2014) Wiener & Thompson, 2015;Paul et al., 2003). In agreement with these findings, we found a modulation of post intervals signals by time in EEG activity that resembled a parieto-occipital p200 and a later fronto-central similar to the LPC. However, only the LPC seemed to be more strongly correlated with time and behavior.
In our results, the LPC had a fronto-central distribution and was inversely correlated with how much shorter the comparison interval was relative to the reference, in agreement with previous studies (Bannier et al., 2019;Lindbergh and Kieffaber, 2013). This inverse relationship between time and LPC amplitude seemed to hold only for intervals shorter than the reference, while intervals longer than the reference had a similar amplitude. This pattern is consistent with the proposal that a decision only needs to be made at the offset of the interval when the duration is shorter than the reference (Wiener, Parikh, Krakow & Coslett, 2018).
Recent proposals have approximated temporal processing with drift-diffusion models of decision-making (Balcı and Simen, 2014;Balcı and Simen, 2016;Simen et al., 2011). When adapted to a temporal discrimination task like ours, these models posit that at the offset of the interval to be judged, evidence accumulates towards one of two thresholds. Our findings are consistent with this proposal, with the pattern of accumulation reflected mainly on the LPC.
Importantly, although the LPC had a temporal distribution similar to other activities commonly associated with a decision, such as the p300 and the centro-parietal positivity (CPP) (O'connell et al., 2012;Twomey et al., 2015), its scalp topography was a little more fronto-central than the p300/CPP. Critically, as mentioned above, the LPC in our findings had a higher amplitude for intervals that are shorter than the reference, consistent with the proposal that this decision process should take place only for shorter than reference intervals (Balcı and Simen, 2014;Balcı and Simen, 2016;Simen et al., 2011).
In conclusion, our results suggest that task goals strongly modulate temporal information encoding in EEG activity. Future studies, using similar approaches, should investigate whether and how different temporal tasks modulate this activity pattern. Sharp-colored filled circles represent the mean JNDs, and bars represent the standard error of the mean.