Electronic data base and evidence based decision making
Information explosion is the agony and ecstasy of the present day science. The internet revolution has opened up the flood gates of data made up of bits and bytes to anyone accessing from anywhere. Thus information has become omnipresent, just in the air, waiting to be accessed. One may assume that this exponential growth in internet and information will provide the best opportunity for data mining. But on the contrary, a chaotic, unorganized tsunami of data that is crashing on to the beaches of scientific civilization just lets us have either “something” or“everything”, seldom the “right thing”. Hitting the bull’s eye in a massive swirl of data depends on how well it is organized and also how methodically it is accessed. The best proof for this is our good old dictionary and encyclopedia. However, as the information architecture in the cyber world is not that conventional, a new skill has to be acquired by one and all.
“Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information on it.” Information that exists outside us has to be processed by a knowledge within us, called as information processing skill. Without this new found ability all of us are sure to catch up with an ailment called as “Information Fatigue Syndrome” – a frustration that arises by staring at a barage of data that makes no sense.In our dental curriculum this information processing skill begins with and alas! ends with the post graduation program – precisely with the submission of the thesis! But we are now at the cross roads of a new philosophy in practice, called as “Evidence Based Decision Making,” (EBDM) in all the health sciences, where we are urged to continue this literature hunt even as clinical practitioners in quest for the appropriate scientific evidence for the treatments offered to the patients.Tracking down information, in order to keep abreast with the latest in the field is not anything new for clinical practice. But the rapid transformation of all the hard copy information into the “Electronic Data Base” and the opinionated informations into well “Researched Evidence” and the submissive patients into wise “decision making partners”, gives novelty to this concept of Evidence Based Decision Making.The evidence based approach educates the clinician to convert the patient’s problems into well framed questions, to search for the right evidence in a methodical fashion, to critically appraise those evidences and categorise them as good, bad and ugly and finally to effectively translate those evidences into clinical solutions that becomes the best care for the patients. In essence, Evidence based approach weds the long separated clinical practice and research. Finally the time has come for research to achieve its prime objective of reaching out to the patients. Though principally evidence based approach is designed for the clinical practice and the patient care, this concept also gives a refreshing insight into different aspects and current status of scientific research to the academic bodies. Through this philosophy we are being educated about various sources of evidence, levels of evidence, quality of evidence, various search engines and also about a pristine method of surfing through this wild electronic database. This can enhance our information processing skills and help us reach the evidence early and effectively. Through this we also stand to gain information about the areas where there is an acute lack of evidence. Thus we may be motivated to do the appropriate research in those areas of lacunae and enrich the repository of evidence, for , after all we are the evidence providers, as well as the end users!
This section will help one
to learn the step by step approach of searching the Electronic Database in the Evidence Based Decision Making way
throw light on the critical areas of Conservative Dentistry and Endodontia that are lacking in evidence
underscore the emphasis on more clinical reasearches to fill in these voids.
Electronic Data Bases – The sources of evidence:
There are two types of evidence based sources – Primary and Secondary. Primary sources are original research publications. Secondary sources, developed by evidence based groups, are systematic analysis of many primary research studies in a particular topic and provide a summarised evidence. Such publications are very useful for the busy practitoners.
Out of many data bases, the MEDLINE is the most reliable and easily accessible multi purpose database for the primary scientific literatures. Created and maintained by the United States National Library of Medicine (NLM) of the National Institutes of Health since 1966, it covers medicine, dentistry, nursing, veterinary medicine, health care services and the preclinical sciences.
The NLM allows free access to the full MEDLINE through three websites called, Pubmed (http://www.ncbi.nlm.nih.gov/ ), Internet Grateful Med (IGM) and the NLM Gateway. Pubmed and Gateway also provide access to pre 1966 journal citations (OLDMEDLINE) and books, serial titles and audio visual resources (LOCATORplus).
A very useful site is MedFetch. MedFetch enables you to repetitively perform a Medline search and receive the results via e-mail. The search is performed weekly or monthly until you delete the search. MedFetch is a web-based software program. With an Automated Medline Query (AMQ), every week you will be e-mailed the 20 most recent citations that match your query from the 11 million citations in MEDLINE and Pre-MEDLINE.
Embase is yet another data base like Medline, which claims more coverage of journals and countries. However this allows access with registration.
Secondary sources are best accessed in the sites maintained by government organizations, academic institutions, evidence based groups and the sites of medical search engines. The following is a long list of sites available in the net.
Centre for Evidence Based Dentistry This is a very exhaustive site on Evidence Based Dentistry. Most recommended site for learning every aspect of EBD. This site also provides evidence based summary of numerous topics categorised topic wise, as well as speciality wise. It also links to other evidence based sites and journals.
Evidence based dentistry is in an extensive form in this site which enables learning EBD as well as searching for evidence
The School of Health and Related Research ScHARR at the University of Sheffield, UK features “Netting the Evidence: A ScHARR introduction to evidence based practice” on the internet, which is an alphabetical compilation of links to numerous evidence based resources.
The Health information Research Unit at McMaster University, in Hamilton, Ontario conducts research in the field of health information science and is dedicated to the generation of new knowledge about the nature of health and clinical information problems, the development of new information resources to support evidence-based health care, and the evaluation of various innovations in overcoming health care information problems.
The library of Ottawa General Hospital provides links to many aspects of evidence based approach.
Medical Matrix is a medical search engine, a paid site requiring registration.
Mediaformatics - Evidents is a dental PICO based search engine for real-time searches of PubMed . Searching for answers to clinical question is made very easy here.
TRIP – Turning Research into Practice- another meta search engine, that is UK-based . Click and see the way the results were displayed on typing a search term. On clicking on each category we shall land up in gold mine of information
SUMsearch is a medical meta-search engine which searches a range of databases andinternet site
Organized Medical Networked Information OMNI is a free catalogue of hand-selected and evaluated Internet resources in Health and Medicine
Critical Appraisal Skills Program CASP is a site that gives all the informations on evidence base that includes the following sections: CASP critical appraisal tools , Open learning resource, E-learning. Sources of evidence , Search filters.
ADEPT This is a workshop and distance learning based programme, about different filters for retrieving the evidence. This comprises of 6 distance learning packages such as Introduction to Research Design, Therapy , Diagnosis, Reviews , Prognosis , Etiology, Economic Evaluations and Guidelines.
The Cochrane Collaboration is an international non-profit and independent organization, dedicated to making up-to-date, accurate information about the effects of healthcare readily available worldwide. It produces and disseminates systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions. The Cochrane Collaboration was founded in 1993 and named after the British epidemiologist, Archie Cochrane. The major product of the Collaboration is the Cochrane Database of Systematic Reviews which is published quarterly as part of the Cochrane Library, where you can browse and search abstracts of reviews free of charge. Abstracts of Cochrane Review and Abstracts of Cochrane Oral Health and Group Reviews are a must – visit aspects of Cochrane database.
Attract In this site when aclinician contacts this site, the evidence is rapidly searched by the answerers in Attract, appraised and summarized onto a side of A4 and then faxed back to the clinician - within 6 hours if necessary.
Database of Abstracts of Reviews of Effectiveness DARE contains summaries of systematic reviews which have met strict quality criteria. Each summary also provides a critical commentary on the quality of the review. The database covers a broad range of health and social care topics and can be used for answering questions about the effects of interventions, as well as for developing guidelines and policy making.
Evidence-Based Dentistry Published by the British Dental Association
Evidence-Based Dental Practice an American journal
The bandolier journal is an independent journal about evidence-based healthcare, Oxford scientists.
Searching EDB the EBDM way:
The invasion of the human mind by the machines, the cyber world gadgets, is amazing. We have a tendency to hold on to the flimsy threads of the web, woven by the information networking, as our life line, in getting informations right from recipes to research. So handling the net may not be a brand new experience for anyone in any age group. Similarly as students, academicians and researchers, we all have had a free hand on the Electronic Data Bases. Whenever in doubt, we type the raw words in the search box and accept whatever spews out of the system happily, even if it takes too much of time or paper. But now,Evidence Based Decision Making brings to the lime light the separate vocabulary and language that is used to communicate effectively with the Electronic Data Bases and also a separate set of protocols in getting what we want out of the congested mass of information. Evidence Based Decision Making encourages the clinician to ask the right question, to get the right answer.A right question is the one that explains precisely the clinical problem that needs to be solved, the methods which are going to be used to solve that problem and the solution that is desired. In EBDM language this is called as the “PICO process”, were P stands for Patient or Problem, I for Intervention, C for Comparison and O for the Outcome. Therefore, a right question will consist of all these four components to convey the query, very specifically. The question can be therapy related, diagnosis related, prognosis related and causation related. The following examples enable a better understanding.
Therapy related question:
P – For an adult patient, with extensive tetracycline stain in the anterior teeth.
I – will an indirect composite veneering
C – compared to a ceramic veneering
O – mask the discoloration effectively?
Diagnosis related question
P –For estimating the working length of a root canal in an adult permanent tooth
I – will an apex locator
C – compared to the radiographic method
O – be able to provide reliable and accurate results?
Prognosis related question
P –In an indirect pulpcapping procedure
I – will intentionally retaining the carious dentin
C – as compared to complete removal of carious dentin
O – affect the vitality of the pulp in the long run?
P –In root canal treatment of a permanent tooth, with an episode of acute apical periodontitis during treatment
I – will use of endomethasone sealer
C – as compared to grossmans sealer
O – result in a peri apical rarefaction later?
Once the right question is ready, in order to get the right answer, it is imperative to know, which type of research study will best answer a certain type of question.Therefore a brief recapitulation of the types of researches and their ranking in the evidence based hierarchy, will make the reading ahead more comprehensible.
A research Recap
Clinical research can be either experimental or observational. Experimental can be controlled and uncontrolled. The controlled experimental studies (where there is a comparison group) are stronger designs in the research family, than the uncontrolled research.
Randomized controlled trial is considered to be the strongest design for the clinical researches. Randomization. blinding, sample size justification, appropriate outcome measures and statistical analysis together reduce the bias in this design.
Observational studies observe a patient at a point of time (Cross sectional) or over a period of time (longitudinal). It can be either prospective (Cohort) or retrospective (Trohoc). In a cohort study, it is known at the outset, whether samples have been exposed or not, to a treatment or possible causal agent and are divided into groups or cohorts (treated or exposed versus non treated or non exposed) on this basis. They are then followed forward in time (prospectively) for years or even decades to see how many in each group develop a particular disease or other outcome. In Case control studies , samples with a particular condition are matched with a group of samples who do not have the disorder and the researchers look back in time to determine the proportion of samples in each group that were exposed to the suspected causal factor. Cross-sectional Studies attempt to establish an association between a possible causal factor and a condition, by determining an exposure to the factor and the condition at the same time. Case reports and case series are often used to describe a condition (usually a rare disorder or a novel aspect of a less rare condition), a new treatment or innovation, or adverse effects of an intervention. This brief knowledge on research designs will be useful in the further sections of the article.
The Research Hierarchy
In simple words, this specifies the ranking order from good design to bad design, in terms of valid and reliable results, with bias control. The highest level of evidence or “gold standard” is the Systematic review and Meta-analysis. These are followed by Randomized controlled studies, cohort studies, case-control studies, and studies not involving human subjects, in this order. The expert opinions rank the lowest in the evidence ladder.
Systematic Reviews provide a summary of individual research studies that have investigated the same phenomenon or question. This scientific technique uses explicit criteria for retrieval, assessment, and synthesis of evidence from individual RCTs and other well-controlled methods.
Meta-analysis is a statistical process commonly used with systematic reviews. It involves combining the statistical analysis and summarizing the results of several individual studies into one analysis. When data from multiple studies are pooled, the sample size and power usually increase.
Research and PICO relationship
Though the randomized control trials and clinical trials enjoy the high pedestal in Evidence Based Decision Making, for some clinical problems they may not be feasible or may not be required at all. Sometimes they may not have been done yet. For example, to assess the success rate of root canal treatment, treatment should be followed up over a period of time (Prospective Cohort study) to observe the prognosis. Here a randomized control trial is not required. Evidence Based Decision Making gives a guideline regarding the best research match for various typology of questions.
- Diagnosis questions : Prospective cohort study
- Prognosis questions : Prospective cohort study
- Therapy questions: RCT
- Causation questions :RCT, cohort or case-control study
Data Search with Pubmed
The following section will concentrate on a systematic and effective way of searching the MEDLINE Electronic Data Base through the PUBMED, the most commonly used primary source for information. What we have in hand now, to clarify our question, is the structured form of the question, the knowledge about the nature of the question and also the knowledge about which type of research papers we have to look for.PUBMED features all its inner space details in a tutorial It would be a satisfying experience to venture into this site. However the tour that we are going to take you through with a sample PICO question will highlight the importance of certain features in PUBMED that will help us in focusing our search.
Sample PICO structured clinical query:
The problem is therapy related
P – For treatment of caries in an adult patient with high caries risk,
I – will ozone therapy,
C – compared to conventional preventive treatment,
O – effectively cessate the progression of lesion?
The type of research to look out for are Meta analysis, Systematic Review, Randomized Controlled trials and Clinical trials in this order of preference. As was mentioned before, the MEDLINE database understands a different vocabulary than that is used in the above PICO question. It is called as the MeSH (Medical Subject Headings) term. This vocabulary contains main headings or index terms, each of which represents a single concept in the biomedical literature. New terms are continuously added and outdated ones are removed by subject specialists as new concepts emerge in the scientific literature. So the first step in the pubmed search is to find out the mesh term for the key topics in the question. Searching the MeSH term is an art by itself. PUBMED provides a MeSH database and MeSH search. Typing the important terms of the question in search box of MeSH database, will suggest relevant MeSH terminologies.Most of the journal articles now provide the key words and the MeSH terms, to make things less complicated.
As the first step in the data search for our clinical query, in the MeSH database of the PUBMED we typed the term ‘Dental Caries’ as in (Fig 3-click to view). The detailed subheadings under ‘Dental caries’ is shown in (Fig 4-click to view). Our question is related to dental caries therapy, we ticked the term ‘therapy’. In the ‘Send to’ pull-down menu we selected ‘Search Box with AND’. On clicking the ‘Send to’ button, the term with specifications appeared in the Search Box ( Fig 5-cllick to view). We did not attempt to search for the Mesh term for ‘ozone therapy’ (Fig 6-click to view). We decided to use the search by the Text word. Text word searches the database by the presence of this term in the title or the article.Thus the search terms [“dental caries/ therapy” AND ozone therapy] were framed. The ‘AND’ here is not simply English. This is called BOOLEAN OPERATOR in the data base search jargon.Boolean logic symbolically represents relationships between entities. There are three Boolean operators. The AND operator is used to retrieve a set in which each article contains all the search terms.The OR operator is used to retrieve documents that contain at least one of the specified search terms. The OR operator is used when you want to pull together articles on similar subjects. Use the NOT operator to exclude the retrieval of terms from your search. All Boolean operators should be typed in upper case.The PUBMED search with this assembly (Fig 7-click to view) yielded 10 articles out of which 3 articles were reviews. Two articles were in French, one was a congress report and two were personal views and letters. Only two RCT studies were obtained. To get the studies that are in the top of the evidence ladder, we can use the LIMIT feature to zoom-in more. LIMITlimits the searches to specific fields, age groups, gender, type of study, Entrez or publication date, a specific language, types of articles, or subsets. We limited the search by selecting Meta analyses (Fig 8-click to view) in the Publication type, English language and Human studies. This search yielded no results (Fig 9-click to view). Then the publication type was changed to Randomized clinical controland we ended up with two articles (Fig 10-click to view). The abstract of both the articles seemed to offer the answer to our question. The answers were promising for the use of the ozone therapy. We decided to obtain the full articles either by tracking down the journals or writing to the authors for reprint. We also have a feature for obtaining the articles in PUBMED which is called as the ‘Loansome DOC service’. As RCT is a recently popularized study design and clinical trials were frequently performed in the earlier times, we limited the search to Clinical trials (Fig 11-click to view) and ended up with the same two articles. However, a very interesting thing was observed. Among the 3 review articles one was a systematic review (Fig 12-click to view) by then Cochrane data base – real evidence! The important message here was, that there is no enough evidence to prove the superior efficacy of the ozone therapy over the other treatment modality! In fact a Meta analysis was not possible as there were no enough RCT in this area! What do we infer from this as a clinician? ..that we cannot promise or expect a cessation of the progression of the carious lesion on using a latest gadget of Ozone therapy, inspite of solitary claims by some studies and the manufacturer. What do we infer as an evidence provider? ..that this is one interesting area of Dental caries management that requires more clinical research. If proven effective, this concept can add to the ultraconservative revolution of Dental caries management. A potential area for a thesis!
More useful features of PUBMED
A very useful operation, called truncation, can be employed when doing text word searches. A truncated term is the first part of a word followed by an asterisk. This feature allows all terms beginning with that part of the word to be searched. For example, “dent*” will find all terms that begin with the letters d-e-n-t, including “dental,” “dentistry,” “dentist” and so on.
One can also access the results for our example PICO very quickly through Clinical Queries link on PUBMED sidebar. This is a search interface to find citations in the areas of Clinical Study Categories, where one can find citations corresponding to a specific clinical study category and Systematic Reviews where one can find citations for systematic reviews, meta-analyses, reviews of clinical trials, evidence-based medicine, consensus development conferences, and guidelines. Four study categories or filters are provided within Clinical Queries: therapy (default), diagnosis, etiology and prognosis. Two emphasis categories or filters are also provided: sensitive search (broad) and specific search (narrow).
Research lacunae and EDB
Now we approached the Electronic Data Base in the similar way, to search for lacunae in evidence for few vital topics, in Conservative Dentistry and Endodontia. The questions were formed on various topics in which we had our own doubts regarding the clinical efficacy or utility. Please note that this list is not a complete one of lacunae in our field. This is only the tip of the iceberg. This section is included in this article mainly to motivate the student and academic community to conduct more refined and relevant clinical studies. It was an awesome experience to find that Meta analyses and Systematic reviews are very few in the secondary evidence based dentistry sources. Searching the primary source in the PUBMED retrieved equally sparse RCTs and clinical trials. Even the prospective and retrospective observational studies relevant to prognosis and diagnosis related questions were very few. We repeated our search in one or more search engines to exclude the possibility of error in our search strategy. Apart from a routine search through the PUBMED, we had also used Clinical query service of PUBMED, Trip Database search engine and the Evident search engine.
The following are the topics where questions were raised and database was searched for answers, but found voids that are potential for research. The interested Post graduates can even consider them for their thesis/ dissertation.
Diagnodent: The search term included the term sensitivity and specificity, a useful MESH term while looking for data related to diagnosis. No evidence based articles were found, but only one Randomized control trial study was found. 22 articles under the category of diagnosis articles were found. Browsing through them we found them to be more of evaluation studies and narrative reviews, which do not stand at the top of evidence hierarchy. A weak information architecture and clinical usefulness of this technology,makes this a potential research subject.
Diagnostic aids used for proximal caries detection: A broad search of the database revealed just one clinical trial and nine other valuation studies. No systematic review or Meta analyses were found. A clinical challenge where we are dependent on just a radiographic image, in all fairness, should have more studies than that is just present now.
Caries risk assessment tests : First rank evidence is severely lacking in the caries risk assessment methods. A simple search for microbial analysis weeded out only two clinical trial articles and about ten diagnostic articles and one causation related and one prognosis related article. However the conclusions of these studies did not singularly and strongly support the relationship of microbial count in saliva as a diagnostic tool for the disease.
Casein (bovine milk phosphor protein): As an anti cariogenic material this material is already prevalent in the clinical practice. But the naked truth is, at this point of time of this search, only one RCT emerged that was done in 2002. Most of the other meager 8 articles found were animal studies and reviews only!
Mineral trioxide aggregate: A material in vogue in clinical practice enjoys popularity in research in terms of 186 articles, but all of them were articles tracked without applying limits for better evidence. On applying evidence limits to human Rct in English language, we found only 13 results. Among these 13, only one evaluated MTA as a pulp capping agent. The others were evaluating MTA as a root end filling material. That is one golden area for research.
Dentin bonding agents: This material as a pulp capping agent is always a subject of controversy between the eastern and western literature. On searching the PUBMED data base using relevant mesh terms, it was interesting to note that only 4 RCT in humans in English were present. More evidence in the form of clinical trials might definitely diffuse this controversy.In spite of all the attention on dentin bonding agents and their fast growing generations, the PUBMED database search for RCT study on humans, in English for the last 5 years, yielded just one study comparing single bottle and two bottle adhesive. This an in vivo study with a good criteria for evaluation. Such studies are welcome in this topic, instead of experimental in vitro studies, the results of which may not correlate well with complex in vivo situations.
Veneer restorations: In the subject of Composite resin restorations, the most happening thing in clinical practice is the indirect composite resin restorations.. With gadgets made available for an in-office set up, it is time to see if there is any strong evidence to support their use. This is what we found searching for information on veneers. Wakiaga J, Brunton P, Silikas N, Glenny AM. Direct versus indirect veneer restorations for intrinsic dental stains. The Cochrane Database of Systematic Reviews 2004, Issue 1. Art. No.: CD004347.pub2. DOI: 10.1002/14651858.CD004347.pub2. This review concludes that “There is no reliable evidence to show a benefit of one type of veneer restoration (direct or indirect) over the other with regard to the longevity of the restoration.
Apex locators: WL estimation with apex locators has been in use both in institutes and clinical practice since very many years. However a search for a good RCT revealed just one study in the year 2000. The reason for this lacuna definitely needs to be explored.
Electric Pulp testers: In an era where lasers are being used for vitality testing in Endodontia, dilemma is still present in the use and interpretation of results with electric pulp testers. The data was mined for information on the diagnostic accuracy of these gadgets over the thermal tests and we landed up with only one article in the year 1999.
Automated Endodontics: Rotary endodontics has taken a permanent place in a clinical set up. A search was made to find its efficacy in all clinical parameters compared to manual instrumentation. However clinical trial search limited to English language and human subjects, resulted in 14 studies. In the past 5 years, studies specifically comparing it with the manual instrumentation resulted in only 4 papers.
Single visit endodontics: This is a debatable concept for too many years, but unfortunately has not initiated quality researches. Trip database revealed only 5 therapy articles and 1 good prognosis article.
Are the ways of Evidence Based Decision Making, the ultimate scale to measure the research status in our fraternity? The answer may not be strongly affirmative. Evidence tracking in evidence based approach, places articles published in English at the top of evidence ladder. Therefore studies in any other language, however good they may be, as per the norms of Evidence Based Decision Making, may get eclipsed. In addition, if a particular article does not mention the type of study design it has utilized (Clinical trial, or RCT, or evaluation study etc), again it gets lost in the search strategy. When searching through the PUBMED database, only journals that are indexed in this will be searched. Those that are not indexed, whatever quality or dedication they may possess, will not contribute to the database. For instance, Indian studies were too conspicuous, simply by their absence in the small search that we had done for this article. This in no way can mean that Indian dentistry has not contributed to the global dental research. The reason may be just that most of our journals are not indexed in the global databases. Is Evidence Based Decision Making going to be the future way of Indian dental practice? Again the answer may not be boldly affirmative. The evidence based practice is in the international scenario, primarily because of the consumer litigation against the dental professionals. That is why the nomenclature information has transformed to evidence, that smells legal! Even though India is shining here and there, most of the dental practices in rural areas are still not sophisticated enough to latch on to a computer system, to update on dentistry and most of our patients still do not have consumer awareness. Therefore Evidence Based Decision Making in India may not be pandemic, at least in the near future. However one off shoot advantage of Evidence Based Decision Making, as we had seen throughout this article is the awareness of research and its influence in clinical practice. In addition, as the dental education in our country is rapidly accelerating, it is time to think of a repository of information for Indian, by Indian. Every academic body in our country can become e- alert and disseminate their researches, electronically through their websites, for national and international benefit. More importantly, the dental post graduate curriculum has to change its outlook towards thesis/ dissertation and treat them as potential database providers than a mere formality to be completed for the partial fulfillment to obtain the Masters degree. If the details of the thesis/ dissertation right from the study objectives to study designs are taken care of, meticulously and scrupulously, the results that emerge are sure to be strong evidences. The curriculum should emphasize on more of clinical trials than in vitro experimental analysis. The data obtained by the experimental in vitro studies do contribute to information congestion that already exists, with at least 700 journals and at least 25,000 publications per year, but, never contribute to qualitative, clinically useful data.Unless and until these lacunae in our professional existence are not eliminated with a refreshed and revolutionary thinking, then the so called lacunae in the research will persist forever.
1. Carounanidy Usha. Getting cornered or Getting ahead? – Evidence Based Dentistry: A view point. JIDA Vol. 72 July 2001.
2. David L Sackett, William M C Rosenberg, J A Muir Gray, R Brian Haynes, and W Scott Richardson. Evidence based medicine: what it is and what it isn't. BMJ 1996; 312: 71-72
3. Forrest JL, Miller SA.Evidence-Based Decision Making in Action:Part 1 - Finding the Best Clinical Evidence. J Contemp Dent Pract 2002 August (3)3: 010-026.
4. Forrest JL, Miller SA. Evidence-Based Decision Making in Action: Part 2 – Evaluating and Applying the Clinical Evidence. J Contemp Dent Pract 2003 February;(4)1:042-052.
5. Richards D, Lawrence A. Evidence based dentistry. Br Dent J 1995; 179(7):270-3.
6. Richard M. Beyers, DDS. Evidence-Based Dentistry:A General Practitioner’s Perspective. J Can Dent Assoc 1999; 65:620-2.
7. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based Medicine: How to Practice and Teach EBM. Second Edition.Churchill Livingstone: Edinburgh, 2000.
8. Sackett DL, Wennberg JE. Choosing the best research design for each question.BMJ. 1997;315:1636.
9. Susan E. Sutherland, DDS .Evidence-based Dentistry: Part I. Getting Started .J Can Dent Assoc 2001; 67:204-6
10. Susan E. Sutherland, DDS. Evidence-based Dentistry: Part II. Searching for Answers to Clinical Questions: How to Use MEDLINE . J Can Dent Assoc 2001; 67:277-80
11. Susan E. Sutherland, DDS and Stephanie Walker, MA, MLS. Evidence-based Dentistry: Part III. Searching for Answers to Clinical Questions: Finding E-vidence on the Internet . J Can Dent Assoc 2001; 67:320-3
12. Susan E. Sutherland, DDS. Evidence-based Dentistry: Part IV. Research Design and Levels of Evidence .J Can Dent Assoc 2001; 67:375-8
13. Sutherland SE. The building blocks of evidence-based dentistry. J Can Dent Assoc 2000; 66(5):241-4.