I was digging through some old files recently and stumbled upon a research paper that I had written for one of my Masters’ program classes. The Masters degree program was M.S. Information Technology Leadership at La Salle University in Philadelphia (http://www.lasalle.edu/information-technology-leadership/) and the course was one of my favorites from the entire program – Legal, Ethical and Social Issues in Computing. What follows is the research paper that I prepared for the class, reproduced in its entirety without a single word changed. The original publication date is December 5, 2000. In the subsequent 17+ years, the situation has only gotten worse – we live in an age where fake news, disinformation campaigns and propaganda wars constantly bombard us and attempt to distort our view of the truth. The information presented below is as relevant in 2018 as it was in 2000. It is incumbent upon us as intellectuals to apply critical thinking and skepticism to information to which we are presented. It is irresponsible of us to accept information as truth on face value. I feel that the information presented below is too important not to share. The future of democracy may very well depend upon our ability to recognize credible information.
NOTE: The sources presented in the original paper are reproduced here as well. However, only the 2nd to last bullet item is still a valid hyperlink (amazing after 17 years!). I have included them all for one simple reason – credibility.
To trust or not to trust …
that is the iQuestion
Assessing Information Credibility on the Internet
December 5, 2000
Copyright © 2000-2018 Jeffrey D. Freedman
Credibility is a quality that makes something trustworthy or believable.
“Few delights can equal the presence of one whom we trust utterly.”
“Few things can help an individual more than to place responsibility on him, and to let him know that YOU trust him.”
Booker T. Washington
“The quality or power of inspiring belief.”
Merriam-Webster Collegiate Dictionary
The Internet is an incredibly vast source of all types of information. Millions upon millions of web pages are available to the researcher for easy perusal. However, there is a wide spectrum of quality and accuracy in the information presented. One of the biggest problems with the Internet today is the proliferation of poor quality or inaccurate information. Whether intentional or otherwise, misinformation or is one of the key risk factors in obtaining research from the Internet.
The lack of quality and accuracy in many of the available information sources on the Internet brings forward the issue of credibility. To the discerning researcher, all Internet information sources become suspect. How is the researcher to use the Internet while being satisfied that the information harvested is indeed credible? The answer lies in the researcher’s ability to critically assess the information source for credibility.
This paper will help lay the foundation for an understanding of the methods by which a discerning researcher can critically assess an Internet information source for credibility.
Types of Credibility
A recent Stanford University study identifies four types of credibility: presumed, reputed, surface and experienced.1 When critically assessing an information source, a researcher should use as many of these types of credibility as possible in drawing their conclusion. This multifaceted approach will lay a solid foundation for the determination of credible Internet information sources.
Presumed credibility is a belief that we hold based on our assumptions.2 Assigning credibility to information found on a government web site because we assume that the government is a publisher that we can trust is an example of presumed credibility. Not for profit organizations, government sites and major news sites (CNN, MSNBC, etc.) all have a strong measure of presumed credibility.
Reputed credibility is a belief in the credibility of an information source based on its reputation.3 If my friend says that an information source is accurate and I believe him, then I am using reputed credibility as the foundation for this belief. There are many web sites that have reviews and user feedback of other sites. In this way, the cumulative opinion of many users can also to reputed credibility.
Surface credibility is a belief in the credibility of an information source based on a cursory inspection.4 If I rapidly scan a web site for gross inaccuracies, find none, and then trust that source, that trust is based on surface credibility.
Experienced credibility is belief in the credibility of an information source based on past experiences with that source.5 If I’ve received accurate information from a source in the past, and I assume that this source will continue to provide me with accurate information in the future, then that belief is founded on experienced credibility.
Assessing Internet Information Credibility
There are a number of critical assessments that can be applied when evaluating Internet information sources for credibility. The remainder of this paper discusses those methods.
Paramount to establishing credibility is the subject of accuracy. Reliable, error-free information is a must. The researchers must draw on their own knowledge but must also apply the methods described below and in the section on “Verifiability” on page 4 [sic].
A strong clue in determining that a source is inaccurate is evidence of contradictory information. As a researcher reads the information, they should be alert for contradictions that appear in the material presented. These contradictions do not necessarily need to appear solely within the web site of this information source. Contradictions can arise based on the researcher’s general knowledge, from other more traditional information sources such as print material, or from other web sites.
Evidence of Quality Control
A key factor in determining accuracy is evidence of quality control. Quality control of Internet information can take several forms. Editorial control and peer reviews are the two most prominent of these. Evidence of both of these methods being instituted on a web site is an even stronger plus for this information source.
Information sources that have an editorial control process display a greater level of credibility. An editorial control process is one in which one or more individuals at the web site are responsible for verifying the accuracy of the information that they are posting. Sites that have an editorial review process in place should have information displayed that describes the process and the people (or types of people) involved.6
Another indication of credibility through quality control is a peer review process. With this method. a group of the author’s peers reviews the material before it is published to the web site. These peers may or may not be full time staff members of the Internet site. The difficulty with this approach is in finding truly qualified “peers”. Peer biographies should be made available so that the researcher can assess their qualifications to adequately review the material.
In order for information presented on the internet to be assessed for accuracy, it must be verifiable. In order to be verifiable, additional supporting material must be presented.
Any methods by which an author has obtained a published result must be documented. For example, if an author claims that using a certain method produced a sample that was 99.99% pure, then that method must be documented. A hyperlink to an existing source with this information can be substituted, if appropriate. By documenting their methods, the author gives the researcher the information necessary to repeat the steps. A researcher savvy in the field could easily identify any errors or omissions in the method or validate its reasonableness. Issues with disclosing methods could arise if the disclosure would reveal confidential information or trade secrets.
Any information presented which is not common knowledge for the intended audience must be further explained. This will allow the researcher to proceed without any ambiguities.
Underlying data presented
Information that was derived from other data must have that data presented so that researchers may draw their own conclusions, if so desired. Again, a researcher savvy in the field could easily identify any errors or omissions in the data or validate its reasonableness.
Claims supported by evidence
It is an extremely important factor in determining credibility, that claims can be crosschecked. Evidence to the accuracy of the information must be readily available for the researcher in order to verify the claims. This evidence must be available from other information sources including traditional print media.
In order for a researcher to appropriately determine credibility of information, it is extremely important that sources of information be clearly identified. All knowledge that is presented that is not original thought, or research must be appropriately credited. Footnotes and/or endnotes are a reasonable means to achieve this goal. In addition, a full bibliography should be available showing all sources both Internet and traditional print media. Internet sources should have a hyperlink to the original research. It is also very important that the constituent research is from reputable sources. The author should have done their own critical assessment of their source’s credibility. If the author has performed this task, then the steps they took should be clearly noted.
Another key factor in determining Internet information credibility is the authenticity of the sources.
Sources should be checked to determine that they indeed exist and are not purely a figment of the author’s desperate imagination. Hyperlinks to Internet sources should be followed and traditional print sources should be verified for existence. An easy method for checking the existence of printed information is by using the search features on the Library of Congress web site (http://catalog.loc.gov).
Source information reproduced unmodified
Simply because a source exists does not guarantee that the information contained in that source has been accurately represented in the final article. The cautious researcher should read the sources and be satisfied that the information has been presented unmodified from the original.
Establishing authority of the author is another key factor in assessing credibility.
The author of the material must be clearly identified. Any information presented without an author listed should be immediately suspect. An author can be one or more persons. but may also be an organization. Organizations claiming authorship of a document frequently have in place a peer review process and editorial control as described in “Evidence of Quality Control” on page 4 [sic]. This can lead a researcher to assign additional credibility, but care must be taken that the organization is reputable and that their quality control procedures are documented.
It is important that author’s biographies are clearly listed. By describing the author’s background, education and expertise the researcher can judge this author’s capability to credibly present information on a particular topic.
Author credentials should also be verified. Information such as professional society affiliations, previous publications or other information that might establish the author’s authority to address the topic should be presented.
An author who won’t entertain questions or comments on their work might have something to hide and should raise a red flag for the discriminating researcher. Therefore, an author’s email address must be listed. Organizations that claim authorship should clearly indicate an email address to which questions and comments should be directed. Additional information could include a mailing address and/or a phone number.
An author’s reputation could very well be an indication of credibility. Other web sites with author reviews or discussion forums can be searched for comments about this author. Positive comments add to this author’s credibility, while negative comments detract from it. Comments from the general public in an unmediated forum must be viewed critically themselves for credibility.
in the same way that establishing authority of the author is important, establishing the authority of the publisher is also a key factor in determining credibility.
The publisher of the material must be clearly identified. Any information presented without a publisher listed should be immediately considered suspect.
Publishers can either be individual, corporate, not for profit or government. Regardless of the type of publisher, their identifying information must be clearly displayed.
Publishers must clearly identify means by which a researcher could contact them if interested. At a bare minimum, an email address must be listed to which questions and comments should be directed. Additional information should include a mailing address and/or a phone number.
A publisher should have information related to their general mission. Clearly stating their organization’s goals could contribute to the credibility of the source.
Publishers that give general corporate information, key staff members (including biographies) and encourage Internet user feedback give a boost to their credibility.
Suitability speaks to a publisher’s appropriateness for publishing information on a particular topic. For example, the National Institute of Health government web site would not be a very credible source of legal information, and the National Bar Association web site would not be a very credible source of medical information. However, if the information presented were reversed, there would be a high degree of credibility.
Just as an author’s reputation should precede him or her, so should a publisher’s. A publisher’s good reputation will increase the credibility of an Internet information source and a bad reputation will have an obviously negative impact.
An author’s ability to maintain objectivity is critical to establishing credibility. Among factors that might cloud an author’s objectivity are conflicts of interest and biases. The researcher must be on guard for distorted material.
Conflict of Interest
Whether paying or not, web site sponsors frequently have an agenda. Some organizations, by their very nature, are not neutral.7 Take for example a labor dispute. The labor union web site might claim that management is taking advantage of the employees. The corporate web site might claim that the employees are using unfair strike tactics to effectively blackmail the company. The truth might be found somewhere in the middle. Then again, it might not. That is the problem with conflicts of interest. The researcher must identify any potential sponsors and try to uncover their real motivation for sponsoring the material.
Advertising is a potential huge conflict of interest for Internet information sources. Some organization with a product to sell, a cause to promote or an opinion to state is paying good money to web sites for advertising real estate. In order for the advertising to be truly effective, it must be targeted to a select audience. This means that you probably won’t find advertising for a food processor on a medical information web site. However, if an article on that site were highlighting the health benefits of red wine, you could see an advertisement for a nice Merlot attached to that article. Now, what was the purpose of that article? Was it to inform the reader of the health benefits, or was it really intended to drive sales of a particular type of wine?
This issue is such a huge factor in establishing credibility, that Consumer Reports magazine does not accept advertising. This is also true for their web site (www.consumerreports.org). This has the effect of causing an increase in the subscription price compared to other magazines, but consumers have the confidence in knowing that their product ratings and reports are unbiased by advertising dollar pressures.
Biases, by their very nature, affect the degree of credibility that information sources engender. Biases come in many forms. Among them are: political, ideological, theological and simple opinions. No matter the reason, biases have a dramatic effect in the quality of information. The researcher should look carefully for biases, whether blatantly obvious or hidden beneath the surface, when considering the issue of credibility.
Another key factor in establishing the credibility of an information source is the currency of the material. Currency is the timeliness of presented information. In order to establish the timeliness of the information, several dates should be presented to the researcher as evidence, the date of writing, date of publication, and the date of last revision should be clearly displayed along with the material.
Provided with this additional information, the researcher is able to determine the appropriateness of the currency as it applies to their application. Different topics of research have different currency needs. For example, an information source that is twenty-five years old would be perfectly acceptable for coverage of Roman history, while a twenty-five year old source covering computer technology would be quite outdated.
As with any traditional media, Internet information that is not presented with appropriate coverage cannot be totally trusted.
The coverage of the material presented must be balanced. Balanced coverage will look at a topic from all available perspectives. This is extremely important if there are dissenting opinions among the experts in the field. Fairness should be the objective. An author who has confidence in their stance, and is able to provide credible evidence to support that view, should not have any qualms about addressing opposition.
When reading information, the astute researcher should be aware of any emotions that are stirred. A style of writing that attempts to draw out the reader’s emotions should be considered suspect. Quality information should be presented in a very matter of fact fashion.
Information that is presented must be presented in its entirety. Imagine discussing the last twenty years of information technology without mentioning Bill Gates. Even though some people might prefer that his name and contributions were not mentioned, in order to do justice to the topic and provide complete coverage it would be a necessity.
Sometimes it is not always possible or desirable for an author to provide complete coverage of a particular topic. There might already exist many fine information sources that express an alternate viewpoint contrary to the author’s. In this case, it is perfectly acceptable for the author to note clearly that their coverage is incomplete and point the researcher to other resources that will round out the topic. If it is not possible to provide complete coverage due to circumstance beyond control of the author, then this should be adequately documented to alert the researcher to this case.
The researcher must attempt to ascertain the purpose that the author and publisher has for presenting this material. It is certainty helpful if the author and publisher have stated their purpose explicitly. However, this is often not the case. Authors and publishers of information that is factual, research and scholarly in nature should clearly state this since these types of information lend themselves well to the subject of credibility.
There are many Internet web sites whose purpose is entertaining, advertising, marketing, sales, advocacy or persuasion. These, by their very nature, are not credible sources of information. Many times, these web sites have the appearance of a more credible information source. It is the job of the researcher to weed these out through careful evaluation.
Critical assessment of Internet information sources is a very important task for researchers concerned about the quality of their material. This task is somewhat more of an art than an exact science. Nonetheless, its importance cannot be denied. It is perhaps a sad commentary on the state of our society that hate groups are attempting to rewrite history, that con artists attempt to dupe the unsuspecting masses, that individual commitment to quality is a hard to find trait, yet this is the reality that we live in. There are many legitimate sources of information available, but finding them is a difficult task. However difficult though, the researcher must apply the criteria discussed in this paper in order for them to establish their own reputation of credibility.
About the Author
Jeff Freedman is a senior strategic technology leader with over 30 years of experience in both information technology and software development, specializing in Software-as-a-Service (SaaS) and Information Security. For the past 17 years, he has held the positions of VP of Technology, CIO & CTO and has been the de facto Chief Information Security Officer (CISO) in these positions as well. Jeff earned an M.S. in Information Technology Leadership from La Salle University and a B.S. in Computer Science from Lock Haven University of Pennsylvania. He is a Certified Information Systems Security Professional (CISSP). He currently runs Triad Information Security, a consulting business focusing on cybersecurity, and can be reached at firstname.lastname@example.org.
- “Quality of Health Information on the Internet”, IEEE-USA, http://www.ieeeusa.org/forum/POSlTlONS/healthnet.html
- “Investors like the Internet, they just don’t trust it”, Sherman Fridman, 8 March 2000, Biz Report, http://www.bizreport.com/news/2000/03/20000308-8.htm
- “Beyond ‘Cool’: Analog Models for Reviewing Digital Resources”, James Rettig. September 1996, ONLINE, http://www.itc.utk.edu/~twelsh/rettig.htm
- “Evaluating Web Resources”, Jan Alexander and Marsha Ann Tate, Last Revision: 21 July 2000, http://www2.widener.edu/Wolfgram-Memorial-Libray/webevaluation/webeval.htm
- “Evaluating web sites & information on the Internet”, Last Updated: 30 August 1999, http://www.namss.org.uk/evalutate.htm
- “Kathy Schrock’s Guide for Educators Critical Evaluation Surveys”, Discovery.com Teacher’s Channel, http://school.discovery.com/schrockguide/eval.html
- “Evaluating Internet Based Information”, Don E. Descy, Ph.D., http://www.lme.manato.msus.edu/class/629/Cred.html
- “Evaluating the Quality of Web-Based Information”, Denise E. Agosto, Last Updated: 3 August 2000, http://www.scils.rutgers.edu/special/dagosto/criteria.html
- “Evaluating Validity and Reliability of Internet Information”, Wayne County Combined General Health District, Last Updated: 24 October 2000, http://www.wayne-health.org/wc_HealthSiteEvaIuation.html
- “Evaluating Web-based Resources: A Practical Perspective”, Angela Elkordy, Cyberlibrarians’ Rest Stop, June 2000, http://www.thelearningsite.net/cyberlibrarian/elibraries/eval.html
- “Checklist for Evaluating Web Resources”, University of Southern Maine Library, http://library.usm.maine.edu/guides/webeval.html
- “Roadmap to the Information Highway: Evaluating Web Sites”, National Institute of Health, http://www.nnlm/nlm/nih.gov/ner/education/info_hwy.html
- “What is Credible Information?”, Donald T. Hawkins, ONLINE, technomonitor, September 1999, http://www.onlineinc.com/onlinemag/OL1999 /technomonitor
- “Quality Management, Certification and Rating of Health Information on the Net with MedCERTAIN: Using a medPlCS/RDF/XML metadata structure for implementing eHealth ethics and creating trust globally”, Gunther Eysenbach, Gabriel Yihune, Kristian Lampe, Phil Cross, Dan Brickley, Journal for Medical Internet Resources (JMIR), March 2000, http://www.jmir.org/2000/3/suppl2/e1
- “Creating a Safety Net – A Proposed Rating Form for Assessing the Quality of Legal Information in Websites”, Alan Robinson, The Journal of Information, Law and Technology, Delivered at the 2nd AustLII Conference on Computerisation of Law via the Internet, Australasian Legal Information Institute (AustLll), University of Technology, Sydney, Australia, 21-23 July 1999, http://elj.warwick.ac.uk/jilt/00-1/robinson.html
- “Establishing and Maintaining Trust in Online Systems”, Claire McInerney, University of Oklahoma, 2000, http://www.ou.edu/class/mcinfo/onlinepaper.htm
- “Criteria for Assessing the Quality of Health Information the Internet”, Mitretek Systems, Health Information Technology Institute, Last Updated: 20 May 1999, http://hitiweb.mitretek.org/iq/onlycriteria.html
- “Credibility and Web Site Design”, Ron Critchfield, Warner Southern College, 1 February 1998, http://www.warner.edu/critchfield/hci/critchfield.html
- “Testing the Surf: Criteria for Evaluating Internet Information Resources”, Alastair C. Smith, 1997, http://info.lib.uh.edu/pr/v8/n3/smith8n3.html
- “Evaluating Quality on the Net”, Hope N. Tillman, Babson College, 30 May 2000, http://www.hopetilman.com/findqual.html
- “Evaluating the Quality of Information on the Internet: Checklist”, http://www.virtualchase.com/quality/checklist.html
- “Practical Steps in Evaluating Internet Resources”, Elizabeth Kirk, Last Updated: 29 November 1999, http://milton.mse.jhu.edu:8001/research/education/practical.html
- “Evaluating and Citing Internet Resources”, Shawn Patterson, University of Detroit Mercy, Last Updated: 3 February 1997, http://www.udmercy.edu/htmls/Academics/library/webpage
- “Evaluating Websites”, http://servercc.oakton.euu/-wittman/find/eval.htm
- “Evaluating Internet Resources for SOSIG”, SOSIG, 2000, http://sosig.esrc.bris.ac.uk/desire/ecrit.html
- “Evaluating Information Found on the Internet”, Elizabeth E. Kirk, Last Updated: April 1 2000, http://milton.mse.jhu.edu:8001/research/education/net.html
- “Evaluation Criteria”, Susan E. Beck, Last Updated: 16 November 2000, New Mexico State University Library, http://lib.nmsu.edu/instruction/evalcrit.html
- “How to Evaluate Web Pages: Questions to ask & Tricks and Strategies to get answers”, Teaching Library Internet Workshops, University of California, Berkeley, Last Updated: 30 August 2000, http://www.lib.berkeley.edu/TeachingLib/Guides/lnternet/EvalQuestions.html
- “Evaluating Internet Research Sources”, Robert Harris, Vanguard University of Southern California, November 17, 1997, http://www.vanguard.edu/rharris/evaIu8it.htm
1 ”Web Credibility Variables”, Stanford University Study, http://www.credibility.org/variables_files
6 “Criteria for Assessing the Quality of Health Information on the Internet”, May 20, 1999, http://hitiweb.mitretek.org/iq/onlycriteria.html
7 “Evaluating Internet Research Sources”, Robert Harris, Vanguard University of California, November 17, 1997, http://www.vanguard.edu/rharris/evaIu8it.htm