Attention Modulates Automatic Movements

Since the majority of human behaviors are performed automatically, it is generally accepted that the attention system has the innate capacity to modulate automatic movements. The present study tests this assumption. Setting no deliberate goals for movement, we required sixteen participants to perform personalized and well-practiced finger-tapping movements in three experiments while focusing their attention on either different component fingers or away from movements. Using cutting-edge pose estimation techniques to quantify tapping trajectory, we showed that attention to movement can disrupt movement automaticity, as indicated by decreased inter-finger and inter-trial temporal coherence; facilitate the attended and inhibit the unattended movements in terms of tapping amplitude; and re-organize the action sequence into distinctive patterns according to the focus of attention. These findings demonstrate compelling evidence that attention can modulate automatic movements and provide an empirical foundation for theories based on such modulation in controlling human behavior.


Introduction
The majority of human behaviors are composed of automatic actions [1][2][3] . Requiring little attentional control, these automatic actions offer the advantage of allowing us to devote our limited attentional resources to other important matters. For example, we can drink a cup of coffee while immersed in deep thinking without worrying about the muscle contractions and relaxations used to execute these movements. However, this does not mean that automatic movements are disconnected from our attention. On the contrary, attention has long been conceptualized as a system of great significance that modulates automatic movements [4][5][6] .
Extensive studies have revealed that focusing attention on body movements remarkably disrupts well-practiced, almost automatic motor skills 7,8 . Taking a vertical jump-and-reach task as an example, focusing attention at the fingertips versus the rungs to be touched lead to decreased maximum vertical jump height 9 . Wulf and colleagues devised a constrained action hypothesis (CAH) to interpret the relationship between attention and automatic movements, proposing that attention can cause deterioration in skill performance by disrupting movement automaticity. However, it should be noted that these studies involved a hidden common confounding factor. Although these actions are generally deemed skillful and automatic, most are guided by deliberate goals. For example, participants were required to jump as high as possible 9 , swim as fast as possible 10 , or throw a dart as accurately as possible 11 . With specific goals to fulfill, it is natural to assume that participants would deliberately control certain aspects of their movements (e.g., force, timing, speed, coordination). As a result, such studies are incapable of elucidating the complex relationship ATTENTION MODULATES AUTOMATIC MOVEMENTS 5 between attention and automatic movement, given it is impossible to conclude that such actions are not entirely voluntary.
Wu and colleagues 12 revealed that only neural activities underlying controlled movements are modulated by attention. For their study, they trained participants to press certain keys in response to visual cues until not interfering a simultaneously performed secondary task. Participants were then asked to perform the same task twice, immediately after training and again after being instructed to use caution in avoiding incorrect keypresses.
Participants took longer to respond in the second case, indicating a goal-induced shift from automatic to controlled keypress responses 12 . This crucial finding stemmed from brain imaging data collected during the study. In the healthy control group (as compared to a group of Parkinson's disease patients), data revealed that attention modulates activity and connectivity in those regions responsible for goal-directed, controlled movements (the dorsolateral prefrontal cortex, anterior cingulate cortex, and rostral supplementary motor area) 12,13 , but not in the region responsible for well-learned, automatic movements (the striatum) 14 ). The results thus pose a challenge for determining whether attention truly modulates automatic movement.
To this end, the present study re-investigates the interaction between attention and automatic movements. Crucially, we did not ask participants to perform automatic movements under any specific goals, given the aforementioned issue of previous studies.
Moreover, the automatic movement chosen for this study was not one newly learned in the lab. Participants were required to repetitively and sequentially tap their fingers at a usual, comfortable pace. Even when attention was directed toward these movements, the lack of a ATTENTION MODULATES AUTOMATIC MOVEMENTS 6 specific goal denied the need for participants to try and control their tapping movements. To quantify the trajectories of tapping fingers, we used a new marker-less video-based pose estimation technique 15 . In the present study, we manipulated attention by asking participants to focus their attention on the movements of either all fingers or a single finger (the index or middle finger), and contrasted these movement-focused conditions with a reference condition in which attention was directed away from the tapping fingers.
To fully elucidate how attention modulates automatic finger-tapping movements, we examined the following three predictions. First, we tested the core proposition of CAH theory 7,8 that attention disrupts the automaticity of movements. Accordingly, we assumed that unattended trajectories of these automatic movements would be duplicated. With regard to the finger-tapping movements, the phase lag between tapping trajectories of any two fingers or trials should be relatively consistent when the movements are not being attended, leading to a higher inter-finger and inter-trial temporal coherence than when they are attended.
Secondly, using tapping amplitude as a measure, we tested a core proposition from the Norman-Shallice theory. This theory, focusing on how attention translates goals into behavior, proposes that attention modulates automatic movements by facilitating attended actions while inhibiting unattended actions. Our primary interest was whether the tapping amplitude would increas for attended fingers while decreasing for unattended fingers. Finally, we performed a pattern analysis on the tapping trajectories of all fingers to examine the prediction that attention focused at different fingers would create distinct patterns of finger-tapping movement. The above predictions were tested in three experiments. Experiments 1 and 2 directed participants' attention to the respective fingers using two different paradigms with ATTENTION MODULATES AUTOMATIC MOVEMENTS 7 objective and subjective measures of attention, respectively, and we expected similar findings in both. For the control, experiment 3 directed attention toward the conceptual label, rather than movement, of respective fingers, and we expected no modulating effect of attention.

Participants
Sixteen college students (7 females) with a mean age of 24.44 ± 0.48 years [hereafter the value following the symbol '±' represents one standard error of the mean (SEM)] took part in experiments 1, 2, and 3 successively over three days spanning eight weeks. The sample size was determined according to previous publications [16][17][18] . Procedures and protocols for this study adhered to tenets of the Declaration of Helsinki and were approved by the review board of the School of Biomedical Sciences and Engineering, Southeast University, China. All participants were naive to the purpose of the experiments, gave prior informed written consent, and were debriefed after each experiment.

Materials
In each experiment, participants were asked to tap their right-hand fingers on a pad fixed to the table and to rest their thumbs at an anchor position marked by a small rubber patch on the pad. Participants' finger-tapping behaviors were recorded by a full HD-camera (TC-UV8000, 20X-zoom, resolution = 1920×1080, sampling rate = 60 Hz) facing the right hand and placed on the table at a distance of 30 cm from the anchor. When participants were informed to tap their fingers using a pure tone, a visual cue was simultaneously presented on the screen in an isolated area. This area was shielded from the sight of participants but ATTENTION MODULATES AUTOMATIC MOVEMENTS   8   continuously monitored by another HD-camera (TC-980S, 12X-zoom, 1920×1080, 60 Hz).
We used a video broadcasting station (model: iRBS-V8, with two video engines) to synchronize videos from these two cameras so we could determine from which frame participants were informed to tap their fingers.

Procedure
Stimulus was presented using Psychophysics Toolbox extension 19 with MATLAB (The MathWorks, Natrick, MA). Upon arrival at the laboratory, participants received an entrance exam in which they had to concurrently perform two tasks. In one task, we required them to tap the fingers (excepting thumbs) of their right hand in a daily practiced sequence of their ensuring familiarity and comfort with the movements. In a simultaneous secondary task, we asked participants to read a text paragraph clearly and loudly. Following prior literature 20 , we surmised that if one could tap his/her fingers fluently without being interrupted by the secondary task, he/she was regarded as qualified for the subsequent experiment. In fact, participants had no difficulty in passing the exam after some practice. In experiment 1 (Fig. 1a.), we examined whether and how these automatic movements are influenced by the focus of directed attention. In a dual-task paradigm 21,22 used as the entrance exam, participants were required to perform the finger-tapping task simultaneously with a secondary, more exhausting task (i.e., a letter-counting task). This paradigm allows us to evaluate the validity of our manipulation of attentional focus 23,24 by measuring the accuracy of the secondary task. Specifically, if participants focused their attention on finger-tapping, the accuracy of the letter-counting task would decrease. The four conditions in experiment 1 included three movement-focused conditions and a reference condition. In the movement-focused condition, attention was directed at the movement of the tapping sequence (sequence-focused condition), the index finger only (index-focused condition), or the middle finger only (middle-focused condition). Under these conditions, participants had to focus their best on movement of the required finger(s), while paying less attention to the letter-counting task. In the reference condition, attention was focused on the sequence of letters (letter-focused condition), and participants were required to count for the appearance of a pre-defined target letter as accurately as possible while neglecting their finger-tapping movements.
At the beginning of each trial, participants were informed of the experimental conditions and target letter (O, G, L, or A, determined randomly). Once their attention was given the required focus, participants pressed the enter key. An initiation beep (500 Hz, 500 ms) indicated participants were to start tapping their fingers until hearing the termination beep (250 Hz, 500 ms). They were advised in advance that any deliberate control of movement was forbidden. During this period, a random sequence of the letters O, G, L, and A were presented at a frequency of 2.5 Hz (stimulus onset asynchrony: 400 ms). Each trial lasted either 6.71 sec (short-duration trial, in which the target letter was presented 3-6 times) or 11.14 sec (long-duration trial, in which the target letter was presented 5-9 times), in order ATTENTION MODULATES AUTOMATIC MOVEMENTS 11 to prevent participants from being able to predict when the trial would terminate. At the end of each trial, participants reported (by pressing keys using their left hands): (1) how many times they saw the target letter, and (2) the amount of attention (ranging from 0 to 100) they had directed to their finger-tapping movements. The next trial began after these responses. A total of 72 trials were segmented into 12 blocks, each corresponding to one condition. In each block, there were three short-duration trials and three long-duration trials. A rest followed every three blocks. Experiment 2 (Fig. 1b.) aimed to replicate the findings of experiment 1 in a single-task paradigm, where participants were instructed to concentrate their efforts on finger-tapping with no secondary task. In line with experiment 1, there were three movement-focused conditions and one reference condition. However, differently and crucially, in the movement-focused conditions, participants were not only told to attend towards, but to forcibly stare at, the particular finger(s), continuously tracking their movements and neglecting the movements of other fingers. In the reference condition, they were told to stare at their static thumbs. The target finger was determined randomly for each trial, with no more than two repetitions. At the end of each trial, participants reported the amount of attention paid to the target finger(s). To save time and alleviate fatigue, the total trial number was reduced to 56, with 7 trials for each duration (short and long) and attention condition. All other aspects were the same as in experiment 1. Experiment 3 (Fig. 1c.) served as a control experiment to examine whether the possible influence of attentional focus on automatic finger-tapping is derived from movement being attended or from its lingering conceptual label in the mind. To this end, we used the ATTENTION MODULATES AUTOMATIC MOVEMENTS 12 dual-task paradigm, as in experiment 1, but replaced the letter-counting task with a label-counting task, in which participants were asked to neglect finger tapping but view a random sequence of finger labels displayed on the screen and count for the target labels. The target label was the "thumb finger" in the reference condition, and the "whole sequence", "index finger'," and "middle finger" in three movement-related conditions. To direct attention away from their finger-tapping movements, participants were instructed to look straight ahead at a screen and devote all attention to the counting task. All labels were written in the participants' native language (i.e., Chinese), and the target label was determined randomly for each trial. At the end of each trial, participants reported: (1) the number of target labels, and (2) the amount of attention they had directed to the movement of related finger(s). Other procedures were the same as those used in experiment 1. There were 56 trials in total, with 7 trials for each duration and attention condition.

Data Analysis
Accuracy of secondary counting task. For experiments 1 and 3, we calculated the accuracy (ACC) of the secondary letter-counting task and label-counting task, respectively, according to an algorithm: ACC = 1-(|Nr-Nt|/Nt), where Nr is the reported number of targets and Nt is the actual number of targets. In the algorithm, missing and false alarms had equal influence on accuracy.
Video-based estimation of tapping trajectories. Participants' finger-tapping movements were extracted using a cutting-edge video-based pose estimation algorithm.
Specifically, by using DeepLabCut 15 running on a Linux (Ubuntu 16.04 LTS) operating ATTENTION MODULATES AUTOMATIC MOVEMENTS 13 system with a graphics processing unit (GPU, NVIDIA TITAN XP, 12 GB of memory), we trained a deep neural network with 50 layers (i.e., ResNet-50) to automatically recognize the coordinates of seventeen key points of interest (Fig. 2a.) for each frame during finger-tapping movements (Fig. 2b.). In the first step, we selected 25 representative frames for each participant from videos recorded during experiment 1, manually marked coordinates of key points of interest in these frames, and generated a total of 400 marked frames. We then randomly assigned 380 frames to the training dataset and the remaining 20 frames as the test dataset. By using default parameter settings, we trained a ResNet-50 model through 800,000 iterations. To evaluate its recognition performance, we calculated the mean difference (in pixels) between the manually marked coordinates and predicted coordinates. It was 3.19 ± 0.07 pixels for the training dataset and 4.65 ± 0.90 pixels for the test dataset, a negligible error considering there were 1500 × 1080 = 1,620,000 pixels within each frame. Thus, we applied this model to extract coordinates for each frame, participant, and experiment, and only imported the vertical (y-axis) trajectories of fingers into subsequent analyses, since fingers were tapped mainly in the vertical direction. For preprocessing, vertical trajectories were segmented (from 30 frames before onset of the initiation beep to 90 frames after onset of the termination beep), duration-normalized (the duration of each trial was normalized to the mean duration of short-or long-duration trials using function interpft.m), and temporally smoothed (sliding window = 6 frames, equaling 100 ms). Finally, outlier values that deviated the previous frame by a minimum distance of 100 pixels were replaced using the interpolation method (inpaint_nans.m).
Defining the temporal and frequency ranges of tapping trajectory. The temporal ATTENTION MODULATES AUTOMATIC MOVEMENTS 14 and frequency ranges for our interests were defined as that during which finger-tapping behaviors were automatically performed without deliberate control. First, as the initiation and termination of tapping behaviors inevitably require deliberate control 4 , we selected the temporal range from 1,000 ms post-onset of the initiation beep to the moment before the onset of the termination beep for further analysis. There were 344 and 609 time points in total for short-duration and long-duration trials, respectively. Second, as automatic finger-tapping behaviors can be deconstructed as repetitive movements of each finger at a constant speed, the frequency range of interest for each participant was defined as the frequency band, which showed the strongest inter-trial temporal coherence. We therefore performed a wavelet transform coherence analysis (WTC, http://www.glaciology.net/wavelet-coherence) for every two trials (Fig. 2c.) to produce a time-by-frequency 2-D coherence map (Fig. 2d.). As a measure of correlation between two time series 25 , each coherence intensity out of the coherence map ranges from 0 to 1, with 1 reflecting complete coherence (absolute phase synchrony) and 0 reflecting no coherence (no phase synchrony) at a given time-frequency point. The coherence intensity was then averaged across the selected time range of interest, trial-durations, fingers, and attention conditions. The mean coherence intensity can now be seen as a function of frequency, among which the frequency with the strongest coherence intensity can be located (findpeaks.m) and named as the tapping rate. The frequency band corresponding to full width at half maximum (FWHM) centered at the tapping rate was determined as the frequency range of interest.

Constant and Fast Tapping Rates between Experiments
As acknowledged, automatic movement operationally refers to those movements that can be performed without interfering with other tasks 4,26 . From this point of view, if finger-tapping movements were automatic, they would not interfere with the secondary task and would remain stable across experiments, even if conducted on separate days or under different task requirements. Given that each participant's finger-tapping behavior consisted of a sequence of repetitive movements with which he/she was familiar, the tapping rate of each participant was hypothesized to be stable across experiments. As expected, the tapping rates from three experiments were highly correlated (0.91 < rs < 0.94, ps < 0.001). Moreover, participants tapped their fingers at a speed as fast as 5.66 HZ ± 0.35 HZ (ranging from 3.07 HZ to 8.43 HZ). It seems impossible for participants to make deliberate plans and intentionally control their movements at such a fast rate 27 . No participants reported attempting to control their finger movements when tapping. This converging evidence supports that participants' finger-tapping movements were automatic and under no intentional control.

Results of Experiment 1
In experiment 1, a one-way repeated measures ANOVA found a significant effect of attentional focus on both letter-counting accuracy (F 3 Fig. 3c.).
These results provide compelling evidence that attention had been effectively directed toward movement in experiment 1. One of the primary goals of experiment 1 was to examine the prediction derived from CAH theory that attention to movement may lead to impaired temporal coherence between tapping trajectory across fingers and across trials. We thus performed two one-way repeated measures ANOVAs on inter-finger coherence and inter-trial coherence, respectively. As shown in Fig. 4a

Results of Experiment 2
In experiment 2, we examined whether the effects found in experiment 1 could be observed when participants' attention was directed toward the target fingers by looking directly at them with no exhausting secondary task. Participants reported that most of their attention (88.27% ± 2.16%) was directed to the target fingers, although they also stated their attention was sometimes distracted by non-target fingers that moved simultaneously with the target finger.
Following the same analyses as experiment 1, the results of experiment 2 first ATTENTION MODULATES AUTOMATIC MOVEMENTS 25 replicated the effect that focusing attention at one's movement can desynchronize the temporal coherence of automatic finger-tapping movements (Fig. 4b.). Specifically, a one-way repeated measures ANOVA found a significant effect of attentional focus on both inter-finger coherence (F 3,45 = 5.31, p < 0.003, ). In addition, as shown in Fig. 6b., such above-chance decoding performance was observed only for cells in the diagonal direction (i.e., correct predictions), while percentage values in most other cells (i.e., incorrect predictions) were significantly below chance level. These results confirm the findings of experiment 1, indicating that attention directed to different fingers can create distinct patterns of finger-tapping movements.

Results of Experiment 3
In experiment 3, we directed participants' attention to conceptual label instead of the movements of their tapping finger(s). Accuracy of the label-counting task was at a ceiling level (94.78% ± 0.85%), and after the experiment, participants reported that they paid little  Fig. 4c.).
Second, one-sample t-tests showed no significant changes in I A values of the three movement-related conditions relative to zero (|t 15  Finally, we performed a pattern analysis. As shown in Fig. 6c., above-chance decoding accuracy was present only in the three movement-related conditions ( t 15  As the decoding accuracy for the reference condition did not exceed chance level, it is likely that the partially remaining capability of decoding in movement-related conditions was due to occasionally spreading attention to the movement of respective fingers during the label-counting task, rather than methodological artifacts 28 29 .
Therefore, the above findings confirmed our hypothesis, indicating that it is attention to movement, rather than to conception, that modulates automatic movement.

Discussion
The present study was, to our best knowledge, the first to demonstrate that attention can modulate automatic movement without deliberate goals. By directing attention to different fingers of an automatic multi-finger repetitive tapping sequence, our main findings were that attention, if focused on movement rather than the conception of different fingers, could lower inter-finger and inter-trial temporal coherence, increase (or decrease) the tapping amplitude of attended (or non-attended) fingers, and create distinct patterns of finger-tapping movements.
As is known, attention is critical for scheduling a variety of motor movements. Even for movements guided by particular goals, it is the attention, rather than the goals, that directly modulates automatic movements 4  All findings discussed above conclude with the basic view that attention can modulate automatic movements, even without specific goals, yet they seem in contrast to a prior neuroimaging study 12 . In data from the latter, the mode of automatic movements processed in the striatum, a region critical in supporting well-learned, automatic movements 5 In sum, we found for the first time that attention can modulate automatic movements without assistance from deliberate goals. Once being directed to perform a sequence of automatic movements, focused attention disrupts movement automaticity, facilitates attended and inhibits unattended movements, and re-organizes the movement sequence into distinct patterns according to the focus of attention.
Author Contributions Statement. X.Z. developed the study concept and designed the study.
Testing and data collection were performed by X.Z. and X.J; X.Z., X.J., and X.Y. performed the data analysis and interpretation under the supervision of W.Z; X.Z. drafted the manuscript, and X.Y. and W.Z. provided critical revisions. All authors approved the final version of the manuscript for submission.
Additional Information. The authors declare no competing interests.  For each frame, the coordinates of these points were automatically recognized using