D. Help for Searchers
Given the great number of bibliographic, factual, and other databases and the variety in their content, means of access, controlled vocabularies and search commands, it is advisable to involve librarians and health information specialists in planning and conducting literature searches. These experts can be especially helpful when planning which databases to search, designing search strategies to address evidence questions, and reviewing search strategies (Institute of Medicine 2011). Many such experts communicate via international networks. One such network of HTA information specialists who work with HTA agencies and related types of organizations around the world comprise the Health Technology Assessment International (HTAi) Interest Sub-Group on Information Resources, which has helped to extend the capacity, processes, methods, and collaboration in HTA. The Cochrane Library Users’ Group provides a forum for exchanging information about using the Cochrane Library, including its usefulness, conducting searches, and format.
More powerful search engines, improved indexing, text word searching, user-friendly interfaces, reference management software, and other advances in health informatics are helping non-expert searchers to retrieve valuable information more effectively and efficiently. As such, they are able to gain a more immediate, hands-on understanding of the scope and quality of literature of interest.
There is an expanding set of resources for supporting search strategies for HTA (Goodman 1993; Institute of Health Economics 2013; Niederstadt 2010; Ormstad 2010; Straus 2011). A recent analysis identified eight methods for determining the optimal retrieval of studies in HTA searches: the capture-recapture technique, obtaining feedback from the commissioner of the HTA report, seeking the disconfirming case, undertaking comparison against a known gold standard, evaluating retrieval of known items, recognizing the law of diminishing returns, specifying a priori stopping rules, and identifying a point of theoretical saturation (Booth 2010). Particularly instructive for clinicians is the series of articles originally published in the Journal of the American Medical Association: Users’ Guides to the Medical Literature, which has been compiled and updated by the Evidence-Based Medicine Working Group (Guyatt 2008).
The NLM continues to improve its MeSH (_Me_dical _S_ubject _H_eadings) controlled vocabulary (used to index and search literature in PubMed and other NLM sources) in HTA and such related fields as health services research, evidence-based medicine, and comparative effectiveness research. Continued work has improved the indexing and searching of RCTs in PubMed and other bibliographic databases (Dickersin 1994; Glanville 2006; Royle 2007; Zhang 2006). Most bibliographic and factual databases have user-friendly tutorials, search engines, and other searching tools that are increasingly standard and familiar to expert and non-expert searchers.
The search for pertinent existing evidence is normally one of the first major tasks of an HTA, and should be planned accordingly. Costs associated with evidence searches can be significant, coming in the form of staff time and acquisition of literature, data sets, and other documentation. Although access to PubMed and other government-source databases is generally free or inexpensive, using some specialized scientific and business databases can be more costly. Database vendors offer a variety of packages of databases and pricing algorithms for these. HTA programs of such organizations as ECRI Institute, the BlueCross BlueShield Association Technology Evaluation Center (BCBSA TEC), and Hayes Inc. sell their reports on a subscription basis. (BCBSA TEC reports are also accessible on its web site to any user for private research or study only.) Some market research monographs and other reports oriented for health product companies, investors and other business interests are priced in the thousands of dollars.
References for Chapter VII
Agema WR, Jukema JW, Zwinderman AH, van der Wall EE. A meta-analysis of the angiotensin-converting enzyme gene polymorphism and restenosis after percutaneous transluminal coronary revascularization: evidence for publication bias. Am Heart J. 2002;144(5):760-8. PubMed
Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431-5. PubMed
Canadian Agency for Drugs and Technologies in Health (CADTH), Information Services. Grey Matters: A Practical Search Tool for Evidence-Based Medicine. February 2014. Accessed April 10, 2014 at:http://www.cadth.ca/en/resources/finding-evidence-is/grey-matters.
Chalmers TC, Frank CS, Reitman DR. Minimizing the three stages of publication bias. JAMA. 1990;263(10):1392-5. PubMed
Dickersin K. How important is publication bias? AIDS Educ Prev. 1997;9(suppl 1):15-21.PubMed
Dickersin K, Min YI. Publication bias: the problem that won't go away. Ann N Y Acad Sci. 1993;703,:135-46. PubMed
Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309(6964):1286-91. PubMed | PMC free article
Dundar Y, Dodd S, Dickson R, Walley T, et al. Comparison of conference abstracts and presentations with full-text articles in the health technology assessments of rapidly evolving technologies. Health Technol Assess. 2006;10(5):iii-iv, ix-145. PubMed | Publisher free article
Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991;337(8746):867-72. PubMed
Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7(1):1-76. PubMed
Galandi D, Schwarzer G, Antes G. The demise of the randomised controlled trial: bibliometric study of the German-language health care literature, 1948 to 2004. BMC Med Res Methodol. 2006;6:30. PubMed | PMC free article
Glanville JM, Lefebvre C, Miles JN, Camosso-stefinovic J. How to identify randomized controlled trials in MEDLINE: ten years on. J Med Libr Assoc. 2006;94(2):130-6. PubMed | PMC free article
Gøtzsche PC. Multiple publication of reports of drug trials. Eur J Clin Pharmacol. 1989;36(5):429-32. PubMed
Goodman C. Literature Searching and Evidence Interpretation for Assessing Health Care Practices. Stockholm, Sweden: Swedish Council on Technology Assessment in Health Care; 1993.
Guyatt G, Rennie D, Meade MO, Cook DJ. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice, 2nd ed. New York: McGraw-Hill Education; 2008.
Hall R, de Antueno C, Webber A; Canadian Research Ethics Board. Publication bias in the medical literature: a review by a Canadian Research Ethics Board. Can J Anaesth. 2007;54(5):380-8. PubMed
Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Accessed Sept. 1, 2013 at: http://handbook.cochrane.org.
Hopewell S, Clarke M, Stewart L, Tierney J. Time to publication for results of clinical trials. Cochrane Database Syst Rev. 2007;(2):MR000011. PubMed
Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007;(2):MR000010. PubMed
Institute of Health Economics, Osteba, AUnETS. Health Technology Assessment on the Net International: 2013. Edmonton AB: Institute of Health Economics. 2013. Accessed February 1, 2014 at: http://www.ihe.ca/documents/HTA%20on%20the%20Net%202013.pdf.
Institute of Medicine. Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Press; 2011.
Ioannidis JP. Effect of statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA. 1998;279(4):281-6. PubMed
Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol. 2002;31(1):115-23. PubMed
MacLean CH, Morton SC, Ofman JJ, Roth EA, Shekelle PG. How useful are unpublished data from the Food and Drug Administration in meta-analysis? J Clin Epidemiol. 2003;56(1):44-51. PubMed
Ormstad SS, Isojärvi J. Information retrieval for health technology assessment: standardization of search methods. Int J Technol Assess Health Care. 2010;26(4):359-61. PubMed
Royle P, Waugh N. Literature searching for clinical and cost-effectiveness studies used in health technology assessment reports carried out for the National Institute for Clinical Excellence appraisal system. Health Technol Assess. 2003;7(34):iii, ix-x,1-51. PubMed
Royle PL, Waugh NR. Making literature searches easier: a rapid and sensitive search filter for retrieving randomized controlled trials from PubMed. Diabet Med 2007;24(3):308-11. PubMed
Schöpfel J. Towards a Prague Definition of Grey Literature. GL 12. Twelfth International Conference on Grey Literature. Prague, Czech Republic. December 2010. Accessed November 27, 2013 at: http://www.textrelease.com/images/GL12_Abstract_S1N1.pdf.
Song F, Parekh S, Hooper L, Loke YK, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):iii, ix-xi, 1-193. PubMed
Sterne JA, Egger M, Smith GD. Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis. BMJ. 2001;323(7304):101-5. PubMed | PMC free article
Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomized controlled trials. BMJ. 2011;343:d4002. PubMed | PMC free article
Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-Based Medicine: How to Practice and Teach It. 4th ed. New York, NY: Churchill Livingstone Elsevier, 2011.
Wilkins T, Gillies RA, Davies K. EMBASE versus MEDLINE for family medicine searches: can MEDLINE searches find the forest or a tree? Can Fam Physician. 2005;51:848-9. PubMed | PMC free article
Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database--update and key issues. N Engl J Med. 2011;364(9):852-60. PubMed | PMC free article
Zhang L, Ajiferuke I, Sampson M. search strategies to identify randomized controlled trials in MEDLINE. BMC Med Res Methodol. 2006;6:23. PubMed | PMC free article