The Sheridan Libraries

Using AI Tools for Research

Guide for finding, evaluating, and using AI tools for academic research

AI tools for research

  • Ithaka S+R provides a Generative AI Product Tracker: Generative AI Product Tracker - Ithaka S+R to track generative AI products for postsecondary faculty or students. The list has been updated regularly.

  • Librarians at McGill University developed a ROBOT method Guides: Artificial Intelligence: AI Literacy to evaluate AI tools according to the following criteria: Reliability, Objective, Bias, Ownership, and Type (Hervieux, S. & Wheatley, A. (2020). The ROBOT test [Evaluation tool]. The LibrAIry. The ROBOT Test ). The table below is an example to use the ROBOT method to evaluate if ChatGPT is a good AI tool.

Use the ROBOT method to evaluate ChatGPT
ROBOT method ChatGPT

Reliability

  • How reliable is the information available about the AI technology?

  • If it’s not produced by the party responsible for the AI, what are the author’s credentials? Bias?

  • If it is produced by the party responsible for the AI, how much information are they making available?

  • Is information only partially available due to trade secrets?
    How biased is they information that they produce?

According to information provided on OpenAI’s (the company launched ChatGPT) website: “OpenAI’s foundation models, including the models that power ChatGPT, are developed using three primary sources of information: (1) information that is publicly available on the internet, (2) information that we partner with third parties to access, and (3) information that our users or human trainers and researchers provide or generate.” (How ChatGPT and our foundation models are developed: How ChatGPT and our foundation models are developed | OpenAI Help Center )

One of ChatGPT’s primary sources of information is information that is publicly available on the internet. As with any internet search, it is the user's responsibility to validate the sources of information before relying on the results. When using ChatGPT as a source, it's helpful to request supporting links or references to verify the accuracy of its responses and minimize the risk of misinformation or hallucinations. The other two sources are subjective, as the identities of the third parties are not clearly disclosed, and the information provided by users and human trainers could be influenced by bias.

Overall, ChatGPT can integrate information from multiple sources but it is a human user’s job to validate the reliability of information it gives.

Objective

  • What is the goal or objective of the use of AI?

  • What is the goal of sharing information about it?

    • To inform?

    • To convince?

    • To find financial support?

The Introducing ChatGPT blog post from Open AI states that “We’ve trained a model called ChatGPT which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.“ (Introducing ChatGPT Introducing ChatGPT )

ChatGPT is a general purpose AI tool to assist us to access information. If you are looking for an AI tool to help with a specific research task, such as literature search/review, data analysis, or writing, you could look for an AI tool designed for that purpose.

Bias

  • What could create bias in the AI technology?

  • Are there ethical issues associated with this?

  • Are bias or ethical issues acknowledged?

    • By the source of information?

    • By the party responsible for the AI?

    • By its users?

 ChatGPT exhibits biases and limitations due to its training data and modeling techniques. These biases arise due to the presence of socially undesirable patterns, stereotypes, and imbalances in the large-scale datasets sourced from publicly available text, which reflects both societal norms and disparities. Additionally, the model sometimes generates inaccurate or nonsensical responses ("hallucinations") and performs poorly on tasks requiring high domain-specific expertise. OpenAI has implemented safeguards to reduce harmful or biased outputs, but complete elimination of bias remains a challenge. The model’s knowledge cutoff (September 2021) further limits its ability to provide current and up-to-date answers. (OpenAI GPT-4 GPT-4 )

Owner

  • Who is the owner or developer of the AI technology?

  • Who is responsible for it?

    • Is it a private company?

    • The government?

    • A think tank or research group?

  • Who has access to it?

  • Who can use it?

 ChatGPT is developed and owned by OpenAI, an private company. Anyone can have free access by creating an account at ChatGPT . Users can sign up for paid version to access the premier features of ChatGPT, including faster respond time and advanced features (more details here: ChatGPT Pricing ).

Type

  • Which subtype of AI is it?

  • Is the technology theoretical or applied?

  • What kind of information system does it rely on?

  • Does it rely on human intervention?

 ChatGPT is a subtype of Natural Language Processing (NLP), specifically a large language model (LLM) and generative AI system. It is an applied AI technology that’s in active use for practical applications such as answering questions, drafting content, assisting with coding, creative writing, learning, and more. Human intervention happens during development (training, fine-tuning, and safety alignment), but there is no direct human intervention during use. (Introducing ChatGPT Introducing ChatGPT ; OpenAI GPT-4 GPT-4)
  • Last Updated: Apr 25, 2025 5:19 PM
  • URL: https://guides.library.jhu.edu/c.php?g=1465762
  • Print Page
Subjects: Research Tools