Evaluating Journal Articles
The steps below will help you limit your library catalog search to articles that are peer reviewed.
For an article to be peer reviewed, it means it was sent to other experts in the field who approved of the article and made sure it conformed to best research practices. That's why peer reviewed articles are usually better sources than non-peer reviewed articles. However, you are still responsible for critically reading and evaluating the articles that you cite.
- You can start from the library website at library.carolinau.edu.
- At the top of the page, there is a search bar that says "Search library resources for..."
- Type your keywords into the search bar and a combination of results from most library databases and our physical collection will come up
- Now, you need to narrow down your results since you are only looking for authoritative scholarly articles
- On the left side of the screen, there is a sidebar with ways to limit your search:
- Under Source Types, check the box next to Academic Journals to limit your search to journal articles:
- Under Limit To, click on the link that says "Show More":
- This will bring up a window with more search options. Check the box that says "Scholarly (Peer Reviewed) Journals", then click the yellow "Search" button at the top right of the window:
- Now your search results are limited to articles from academic, peer-reviewed journals.
Often, you have to rely on your own judgment to tell if an article is an authoritative source or not. There's not an exact science to determining the validity of a source, but this page offers some questions that you can ask yourself as guidelines to determine the quality of your article.
- Who are the authors of the article?
- Does the article include their credentials or bios?
- Are they experts in the field?
- If the article states its intended audience, does it say the audience is experts in the field rather than the general public?
- If the audience is not explicitly stated, does the language of the article read as if it is intended for experts?
- Does the article use technical language that only other people in the field would understand?
- Is the article written in a clear, organized way?
- Does it have an abstract?
- Does it have headings that indicate original research, such as Introduction, Method, Results, Discussion?
- Does it have plenty of citations on every page and an extensive list of references?
- Does the article contribute new, relevant information to the field?
- What is the publication date of the article?
- If your field changes quickly, is the article recent enough to be relevant?
- What journal is the article published in? Is it frequently cited in the field?
- What organization publishes the journal? Is it a credible association, like the American Psychological Association? Is it an academic publisher like Wiley or Taylor and Francis? Is it a university press, like Oxford or Cambridge?
Some publishers take advantage of the open access publishing model, which charges authors publication fees instead of charging readers subscription fees, to make money by exploiting authors. These publishers typically accept submissions without putting them through a serious vetting process. They charge very high fees for submitting articles, and then publish those articles without carefully making sure they conform to quality research standards. These publishers produce predatory journals that are driven by profit instead of academic integrity.
Here are some of the basic characteristics of predatory journals:
- Take advantage of the open access model
- Charge authors high fees
- Do not conduct rigorous peer review
- Are not dedicated to producing the highest quality academic research
Here's a quick overview of the concept of predatory publishing with tips for identifying predatory journals:
Predatory journals publish articles that have not been thoroughly vetted and are not up to the standards of respected scholarly research. This means that articles you find from predatory sources likely weren’t conducted according to sound scientific research principles. Articles that don't align with good research practices are not reliable sources of information, so you could end up citing statements that are not verified, which would reflect poorly on your work. Predatory journals are becoming more common and easy to access, so it's important to be able to spot the signs of poor research.
Here are some ideas for how to evaluate a publisher or journal to see if it looks like it could be a predatory journal:
- Try using the list of questions on the Article Evaluation Checklist page of this guide to make sure your article looks like a reliable source.
- Use this checklist from Loyola Marymount University to evaluate journals and publishers.
- Use the BeiNFORMED Checklist from Duke University to evaluate a journal's Number, Fee, Ownership, Review, Membership, and EDitorial board.
- Check your journal title against Beall's List or the List of Predatory Journals, websites that lists suspect journals and publishers.
- Look for high quality open access journals on the Directory of Open Access Journals; if any open access journal isn't listed there, it's probably not reputable.
Let's look at a journal and walk through the process of evaluating it according to LMU's checklist for identifying predatory journals. We'll use the International Journal of Advanced Research in Computer Engineering & Technology (IJARCET).
1. Web Search
The first criteria is a web search for the journal. If we type in the actual title of the journal, it is the first result. We’ll give the journal 3 points.
2. Journal Name
Next: could the journal name be confused with another journal? Let’s type in some of the keywords in a Google search. There is at least one journal - Journal of Computer Science and Technology - from a reputable publisher, Springer, that has a very similar title. We’ll give the journal 2 points for having a similar but distinguishable title.
3. Editorial Board
The next criteria is the editorial board. We can find this information by clicking “Editorial Board” on the IJARCET website. The list of editors includes each one’s name and institutional affiliation. However let’s also note that almost all of the editors are based in India, even though the journal claims to be “international” in its title. Since the title doesn’t match the geography of the editors, let’s give this journal only 2 points for its editorial board.
4. Review Process
Next up is the review process. In the first paragraph of the homepage, the journal is described as “peer-reviewed.” However, nowhere on the website describes the review process or provides any evidence that it is actually peer reviewed. So, we’ll give the journal 2 points for the peer review process.
5. Conflicts of Interest
Next: Does the journal state conflicts of interest? There’s nothing on the website that address conflicts of interest at all, so we’ll give the journal 1 point.
6. Journal Website
The journal’s website itself is the next item on the list. The website is farily easy to navigate, but there is some missing information, as we just saw when looking for the conflicts of interest. Also, the bottom of the website tells us that the site is managed by Wordpress, a free website builder. We would expect a scholarly journal to have a website created by a professional web designer, so we will give this journal 1 point for the website.
7. Revenue Sources
Next up is revenue sources, so we will look around the journal website for information about its business model. We want to find things like subscription or author fees or sponsorship. The publication fee page tells us the authors must pay an online maintenance charge, which is different for authors in India than for authors in other countries. The details about the full business model are sparse, and it is again suspicious for a journal that calls itself international to place a priority on a specific country. We’ll give IJARCET 2 points.
8. Journal Archive
The next item of interest is the journal archive. IJARCET includes an archive of issues with each article available to download in PDF format. However, we can see gaps in the issues, like issues 8 and 9 from volume 9 are not available. We will give it 2 points for that.
9. Publishing Schedule
The publishing schedule can be determined from looking at the list of archived issues on the left side of the screen, but the publishing schedule is not specifically stated. Also, we can see that some of the issues from 2020 are missing. Finally, the top of the website says they are calling for papers for the December 2020 issue, which has already passed. This indicates that the journal doesn’t consistently publish issues and may not have enough submissions to publish. We’ll give the journal 1 point.
10. Author Fees
The author fees are clearly stated on the Publication Fee page, so the journal gets 3 points for that criteria.
11. Copyright Information
The next criteria is copyright Information. There is no statement about copyright on the website, even though the journal says it is open access. Most open acces journals will state that the articles are under a Creative Commons license. We’ll give the journal 1 point.
12. Journal Index
Next, we look at the journal indexing. Good journals are indexed in databases like Web of Science, ATLA, or Academic Search, and will state where they are indexed on their websites. This journal states that it’s indexed on Bing, Yahoo, Google, and Academia.edu. None of these are scholarly databases, so we’ll give the journal 1 point for that.
13. Access to Journal Articles
Access to journal articles is the next criteria, and we can easily access full text articles from the journal’s archives, so we will give it 3 points.
14. Number of Articles Published
For number of articles published, we can tell the journal has a pretty long publication history the includes more than 10 articles. However, if we look at issues from early years like 2014, those issues had a lot of articles. The most recent issue from July 2020, which is a year ago, only includes one article and still has a note that more papers from that issue will be available soon. Since one article in an issue is not enough for a standard journal, let’s give the journal 1 point.
15. Web Search for Publisher & 16. Publisher Information
Now we move onto evaluating the publisher. There is no information about the publisher on this website. Normally publisher information would be found at least in the footer at the bottom of the page, but this website only includes the title of the journal and states that it’s powered by WordPress. Since we can’t do a search for an unknown publisher, we’ll give the journal 0 points for web search for the publisher and 1 point for publisher information, since there’s no information available.
This adds up to a rating of 26, which the guide would call poor. It’s a good idea to avoid citing articles from this journal, because it does not display the rigorous standards of scholarship that are expected for academic work. Ideally, a journal would get 3 points in every category.
Journal metrics are measurements of the quality of a journal, usually based on how frequently a journal is cited in the field. Note that in our example, IJARCET states its ICV, or Index Copernicus Value. ICV is not an accepted or standard journal metric. Here are a few of the most common accepted journal metrics, along with information on how to find them for a journal:
Journal Impact Factor
Journal Impact Factor is the oldest and most widely used journal metric. It measures journal impact by dividing the number of citations in the past year by the total number of articles published in the two years before. A 5-year impact factor is similar, but divides the number of citations in the past five years by the number of articles published in the last five years.
Journal Impact Factor can usually be found on the publisher’s journal information page or from doing a Google search of the journal title and the words “impact factor.”
Eigenfactor is based on citations from the past five years and uses a complex algorithm to relate the wide spectrum of scholarly journals to each other. It takes into account things like discipline, where a mathematics journal might reasonably get a different number of citations than a chemistry journal.
Eigenfactor can be found by searching for a journal on eigenfactor.org.
SCImago Journal Ranking
SCImago ranks journals according to different factors, including the prestige of journals that cite the cited journal.
SCImago journal ranking information, including charts showing SJR and citation level history, can be found on scimagojr.com.
An H-Index’s h is the number of articles an author has published that have been cited at least h times. So if an author has published 7 articles and they have been cited 10, 9, 4, 4, 3, and 2 times, then the H-Index is 4 because 4 of the articles have been cited 4 or more times. While H-Index is usually an author metric, it can also be calculated for journals.
Journal H-Indexes are displayed in searches on scimagojr.com as well.
Now it's your turn! Take a look at the journal websites listed below and use resources listed on this page to decide if you would want to cite articles from it:
Here are some more resources if you are interested in reading more on the issue of predatory journals and journal evaluation:
- Predatory Publishing from George Washington University
- Understanding Predatory Publishers from Iowa State University
- Scholarly Metrics from Johns Hopkins University
- Citation Contamination: References to Predatory Journals in the Legitimate Scientific Literature from The Scholarly Kitchen
- Peer Review Video - a video from Credo Learning that talks about the peer reveiw process
- Evaluating Sources Video - another Credo Learning video that focuses on evaluating sources
- How to Evaluate Journal Articles - a guide from Colorado State University with more tips and questions to ask in evaluating journal articles
- Peer-Reviewed Articles - a resource from San Diego State University for comparing scholarly articles to popular ones
- Scholarly and Popular Materials - a guide from NC State University comparing a scholarly journal, a popular magazine, and a trade publication