A step-by-step guide to systematically identify all relevant animal studies

Before starting a new animal experiment, thorough analysis of previously performed experiments is essential from a scientific as well as from an ethical point of view. The method that is most suitable to carry out such a thorough analysis of the literature is a systematic review (SR). An essential first step in an SR is to search and find all potentially relevant studies. It is important to include all available evidence in an SR to minimize bias and reduce hampered interpretation of experimental outcomes. Despite the recent development of search filters to find animal studies in PubMed and EMBASE, searching for all available animal studies remains a challenge. Available guidelines from the clinical field cannot be copied directly to the situation within animal research, and although there are plenty of books and courses on searching the literature, there is no compact guide available to search and find relevant animal studies. Therefore, in order to facilitate a structured, thorough and transparent search for animal studies (in both preclinical and fundamental science), an easy-to-use, step-by-step guide was prepared and optimized using feedback from scientists in the field of animal experimentation. The step-by-step guide will assist scientists in performing a comprehensive literature search and, consequently, improve the scientific quality of the resulting review and prevent unnecessary animal use in the future.

Prior to setting up a new animal experiment, thorough analysis of previously performed experiments is paramount from both the scientific and ethical viewpoints. Through such analysis, the maximum amount of available information can be derived from previous work on the research topic and, as a result, unnecessary duplication of experiments can be prevented. Moreover, new insights that may arise from aggregating earlier work can be used to improve the design of future animal experiments. The method that is most suitable to carry out a thorough analysis of the literature is a systematic review (SR). 1 An SR implies that a review has been prepared using a systematic approach to minimize biases and random errors, and that the components of the approach are documented in a 'Materials and methods' section. 2 An essential first step in an SR is to retrieve all potentially relevant studies. 1 This requires adequate formulation of the research question, followed by a comprehensive search strategy. A comprehensive search strategy strongly underlies the quality of any literature review. 3 Not finding all relevant studies concerning a specific topic or omitting potentially relevant studies from a literature review may result in bias and, consequently, false or imprecise conclusions. When the omitted experiments are a random sample of all conducted experiments, only the precision of the outcome measure is decreased. However, when a particular selection of, for example, studies with non-significant or 'negative' results is not found and therefore not included, the SR will be biased towards a positive result. This so-called reporting bias may increase potential hazards involved in translating experimental outcomes to possible clinical benefits. 4 -6 A systematic search may reduce the effect of reporting bias, but does not completely solve the problem. Some reporting bias and more specifically publication bias will remain and should be further investigated when analysing the results. A central register of performed and planned experiments as well as publishing the negative results of a research project might further diminish the problem of publication bias. 5 In contrast to the clinical field, where writing SRs and thus executing comprehensive search strategies are common practice, this is not the case within preclinical animal research. 7 -8 This is surprising, since animal studies are often used as the foundation for clinical trials. One of the reasons may be that this approach of formally summarizing the results of previously performed experiments is not yet well known to many biomedical researchers designing animal experiments. Moreover, available guidelines from the clinical field cannot be copied directly to the situation within animal research. In addition, searching for and finding all the available literature on animal studies is not an easy task 9 , even though attempts have been made to make the search process easier. Hooijmans et al. 10 and De Vries et al. 11 describe filters for PubMed and EMBASE, respectively, to ease the process of finding all animal studies in these major biomedical databases.
To facilitate a structured, thorough and transparent search to identify all relevant animal studies (in both preclinical and fundamental science) concerning a specific research question, we developed an easy-to-use, step-by-step search guide which was optimized through the feedback from scientists in the field of animal experimentation. This step-by-step guide will assist scientists in performing a comprehensive literature search and, consequently, improve the scientific quality of the resulting review and prevent unnecessary animal use in the future.
Effective methodology and approach: the step-by-step guide To prepare this step-by-step guide, we made use of the literature, 12 -16 the Cochrane Handbook for Systematic Reviews of Interventions 3 and experts in the field of laboratory animal science. The experts in the field of laboratory animal science were asked to test the step-by-step guide for ease of use and to evaluate the clarity of the text. They reported their findings on an evaluation sheet with prespecified questions. The step-by-step guide was optimized through feedback from these experts.
The basic steps on how to design and carry out comprehensive search strategies to identify potentially relevant animal studies on a specific research question are presented in Table 1 and explained more fully in the following sections.

Formulate research question
A focused, well-formulated research question is a prerequisite for designing an optimal search strategy.
Since the main components of the research question determine the structure and the sequence of the searches, it is important to carefully identify these components. Within animal research, a specific research question generally contains the following components: (1) intervention/exposure; (2) disease of interest/health problem; (3) animal/animal species/population studied; and (4) outcome measures. A well-formulated research question could then be: 'What is the effect of [intervention/exposure] on [outcome measures] in Table 1 Basic steps on how to design and carry out a comprehensive search strategy to identify potentially relevant animal studies on a specific research topic Step

Leenaars et al.
Step-by-step guide to identify animal studies [animal/animal species/population studied] for [disease of interest/health problem]?' Formulation of a focused research question is also important for building an efficient (sensitive) and low numbers needed to read search strategy. 12 Parallels can be found within the clinical setting where the acronym PICO (Participants, Interventions, Comparisons and Outcomes) is often used to determine the critical aspects of a clinical question in intervention studies. 16 Fundamental science could also benefit from this step-by-step search guide, because the search strategy remains the same, but the building blocks for the research question may be a bit different (e.g. it is dependent on the scientific field studied). Although other types of search components may be relevant when formulating a specific research question in fundamental science, the translation of the research question into a search strategy is similar.

Identify appropriate databases and other sources to search
To identify as many relevant papers as possible, searching in more than one database is recommended. Within animal research, more than 100 databases and websites are available containing specific information on laboratory animal science. 9,17 This information is not necessarily available in the more regularly used major biomedical databases. It may be worthwhile checking for potentially relevant papers in a few of these databases as well. Different bibliographic databases vary in type of publications included and the periods of time covered. 14 Besides electronic bibliographic databases, other sources could also be used to identify relevant studies, such as reference lists in retrieved papers. 3

Transform research question into search strategy
Once the research question is defined and the databases are selected, the search strategy can be prepared. Designing a search strategy for a specific database is a complex task. For building a comprehensive search strategy, one needs to be familiar with the specific search terms about the research topic and with the search engine's specific possibilities of searching. Literature searches can be difficult without a basic knowledge of the way information is organized and indexed. Basic and advanced courses in the use of major electronic bibliographic databases are often available at the institutional library. Taking such a course is strongly recommended. It is also advisable to involve an information specialist from the library for technical assistance when designing comprehensive search strategies. In Table 2, an overview of the detailed steps and an example of how to transform a research question into a search strategy in PubMed are given. These steps are more fully described and explained in Section 3. [MeSH]), also papers in which, for example the entry term 'presenile dementia' is used, are found).
PubMed provides the MeSH database to explore relevant terminology and synonyms and to see the relationships between the terms. This can help to find the appropriate terminology.
For identification of MeSH terms and documentation of the search strategy, it can be practical to use a word processor, such as MS Word. The potentially relevant MeSH terms, including synonyms, e.g. entry terms, can be to search term in search bar) and assessing how the retrieved results cover the subject. When, on evaluation of the search results, it turns out that too many irrelevant studies are identified, or that the search resulted in too few hits, reformulation of the search terms is necessary.
B2. Identify free-text terms for SC1 Besides MeSH terms, free-text terms also need to be identified when designing a comprehensive search strategy. It is important to use freetext words in addition to MeSH terms because not all papers that are searched through PubMed have already been indexed. It may take 2-6 months until indexing is finished (L. Bowes, US National Library of Medicine, personal communication). Recently published papers may therefore not yet be indexed and consequently be missed. Another reason for using free-text words is that papers are not always correctly indexed.
To be complete, all possible relevant free-text terms need to be identified. It is practical to start with the MeSH terms and entry terms mentioned in the MeSH database of PubMed. Subsequently synonyms and related terms can be identified. Relevant articles (often reviews), Scopus, Google, etc. can be used for this purpose. Here, common text words may be identified as well as the index terms that the indexers have assigned to the articles. Free-text words should include all synonyms in singular and plural forms, UK and US English spelling, trade names, relevant abbreviations and trademarks. For example, when searching for 'docosahexaenoic acid' a frequently used abbreviation is DHA (Table 2, step B). Common spelling errors should also be covered; for example, ginko biloba for ginkgo biloba.
To economize the search string, truncation [adding Ã to a word stem] is often used in combination with free-text words. Truncation allows finding all the terms that begin with a given text string. For example, if you search statistic Ã , PubMed will simultaneously retrieve words such as statistic, statistics, statistical, etc. However, unwisely truncated terms may dramatically increase the number of unwanted results. The advice is to handle truncation with care and always check the search details for relevance.
To economize the search strategy, you can add [tiab] after a search term in PubMed and consequently your search will be restricted to the title and abstract of article citations and other less useful parts of the citation (such as author names and addresses) will not be searched. When [tw] is added after a search term, PubMed will search for that specific word in all subject-related fields in the article citation. This may result in irrelevant papers, for example, where the author's name is also the name of an intervention.
Free-text words should also be checked for appropriateness. The appropriateness of free-text terms can be evaluated by performing a PubMed search (add [tiab] or [tw] to the free-text terms in the search bar) and assess how the retrieved results cover the subject, how many hits there are and the context in the search history.
For identification of free-text terms and documentation of the search strategy, it can again be practical to use a word processor, such as MS Word. The finally selected free-text terms used in the search are documented as well.
B3. Combine MeSH terms and free-text terms for SC1 After collecting appropriate MeSH terms and free-text words for SC1, all these terms need to be combined into one search string. This can be done by combining all search terms, MeSH terms þ free-text terms, with the Boolean Operator 'OR'. When this search string is used in PubMed, it results in search results for SC1. The search string for SC1 is saved in order to be used in future search actions and for reporting purposes.
Since the aim of the comprehensive search strategy is to retrieve as many relevant studies as possible, high sensitivity is necessary. Sensitivity, in this context, refers to a search that identifies as many research articles as possible that are potentially relevant. This may result in low precision and, consequently, in many hits. This is common to a comprehensive search strategy.
D. Combine search results for SC1 and SC2. After retrieving search results for SC1 and search results for SC2, they have to be combined into one search string in order to retrieve potentially relevant papers. Papers containing both SC1 and SC2 search results are then identified. The Boolean operator 'AND' can be used to combine search results for SC1 AND SC2 from the search history.
E. Evaluate search results. When developing a comprehensive search strategy, there needs to be a balance between comprehensiveness and precision. To be comprehensive, you need high sensitivity. As a result, the precision may be low and the number of hits may be high. This is not necessarily a problem since there is a third and maybe a fourth SC to combine the search result with to obtain more specific results and less hits.
F. Repeat steps B1 -B3 for the remaining SCs (i.e. SC3 and occasionally SC4). To prepare a search string for SC3 (animal/animal species/population studied) steps B1 -B3 can be repeated. To identify all animal studies in PubMed, Hooijmans et al. 10 designed an animal search filter, the so-called 'Animal filter'. The 'Animal filter' is freely available from the publisher's website and can be copied and pasted into the search bar in PubMed. Use of this filter is easy and improves the search efficiency.
As mentioned before, not all components of the research question lend themselves well to searching. For example, if SC4 is an outcome measure that is often not well described in the title or abstract of an article and is not well indexed with MeSH terms, then this component should not be included in the search strategy to prevent missing relevant studies.
G. Combine search results of all the separate SCs. In a final search, all separately developed search strings for each SC need to be combined to retrieve only the potentially relevant studies (Figure 1). This can be done by combining all the separate search results for each SC from the search history in PubMed using the Boolean operator 'AND'. This results in final search string: SC1 AND SC2 AND SC3 (AND SC4). The final search string can be saved in a Word file and in PubMed for future use, to update the search results and to be able to report the exact search string in the review published. PubMed also has the functionality to automatically apply a specifically saved search string to most recently published or indexed articles. The reference details of the articles identified in this way will be sent to the researcher by email.
H. Evaluate search results. Similar to step E, the final search results of search string SC1 AND SC2 AND SC3 (AND SC4) have to be evaluated for relevance. In case the search result SC1 AND SC2 AND SC3 (AND SC4) retrieves too many hits, the terminology in the search string can be adjusted and narrower terms can be selected. In order to be as complete as possible, it is quite usual to have 3000 -5000 hits. However, one should not be set back by 8000 -12,000 hits at this stage. In Section 5, we describe how to deal with these numbers of hits. When a search string is too narrow, low numbers of hits will be found (e.g. less than 300). Broader terms should be selected in order to be as complete as possible in a search. Developing search strategies is an iterative process in which the terms used are modified, based on what has already been retrieved. This means that based on the number of hits and their relevance, the search terms can be adjusted to being broader or narrower.
To restrict the number of hits in the search strategy, the use of limits in PubMed may seem attractive. However, limits should be used with caution to prevent missing potentially relevant studies and consequently introducing potential bias. From clinical literature reviews, for example, it is known that to prevent selection bias, there should be no restriction on the language of the identified studies, because 'negative results' are often published in non-English journals. 13 There should be a rationale for applying limits to a search strategy and it should be indicated in the text of the review.

Transform the research question into a search strategy for other databases
Besides through PubMed, other databases should be used to identify as many relevant papers as possible. The search strategy for PubMed described in Section 3.1 can be (slightly) adjusted and used to design a search strategy for other databases and search engines. One should be aware that different databases use different levels of keyword systems and different levels when indexing material, if any. Again it is advisable to involve an information specialist from the library when designing comprehensive search strategies, especially when you are not experienced in searching in specific databases.

EMBASE.
A major biomedical database that also includes standardized subject terms is EMBASE. In EMBASE, the standardized subject terms are called EMTREE terms. A similar approach to find standardized subject terms and free-text words as used in Table 2

Collect search results of all databases and remove duplicate citations
After developing a search strategy for each database and collecting the citations from the separate databases, the search results need to be combined into one file in the reference program. Citations that appear more than once can then be removed. Duplicate citations often appear because databases can encompass the same journals. After removing the duplicate citations, a list of potentially relevant papers is created.

Identify potentially relevant papers
The list of the potentially relevant papers needs to be screened for relevance of the studies with regard to the research question. The first screening can be done very quickly using only the title and abstract of the studies. To prevent bias in the selection process, it is very useful to have two observers screening the papers independently for relevance. The criteria used for the first screening are based on the SCs. For example, when the citation clearly does not describe an animal study, is not a primary study or is not studying the disease of interest, these citations can be removed since these are not relevant to the research question. Only clearly irrelevant citations should be removed (i.e. one should be over inclusive to prevent incorrect removal at this stage).
The citations resulting from the first screening undergo a second screening when writing an SR. The second screening is based on predefined in-and exclusion criteria and is also performed by two independent reviewers. Throughout the entire screening process for potentially relevant papers the reasons for removal of citations should be documented and reported to facilitate transparency and to make it possible for an interested reader to independently examine accuracy of study removal (Figure 2).

Documenting and reporting the search process
The search processes should be documented in enough detail to ensure correct reporting in the (systematic) review and for the reproducibility of the searches in all databases used. Detailed documenting of the search process is also important to keep track of identified studies and to be able to update the search at a later state. Reporting criteria for the search process in SRs in human research are available. 15 In short, to be systematic, explicit and transparent, the scientist should always report: (1) all databases and other sources searched; (2) the dates of the last search for each database and the period searched; (3) full search strategies (including all search terms) for each database; and (4) any language or publication status restrictions used. An example of reporting on search results and reasons of exclusion of studies is given in Figure 2.

Concluding remarks
A comprehensive search strategy is a key element to ensure the quality of a (systematic) review and the validity of its findings. 3 Searching for pertinent literature on animal studies can be very difficult. 9 To facilitate a structured, thorough and transparent search process to identify potentially relevant animal studies concerning a specific research question (in both preclinical and fundamental science), we developed an easy-to-use, step-by-step search guide. The practical guide presented in this paper is meant to assist and stimulate scientists in performing a comprehensive literature search and adequately document and report on the search process. Transparency on the search process and adequate reporting makes it possible for others reading the review to judge the thoroughness of the search, and thereby the potential of bias in the review. This practical guide cannot be used to identify unpublished studies. Leaving out unpublished studies may result in an overestimation of the effect size because unpublished studies often contain negative data. 5 It should be noted that this step-by-step guide should be used with caution, because this guide is not exhaustive. It is highly recommended to always involve an information specialist for assistance when designing comprehensive search strategies.