GPT-3.5 vs GPT-4: Evaluating ChatGPT’s Reasoning Performance in Zero-shot Learning


May 23 , 2023 read

Discover the first version of our scientific publication "GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance in Zero-shot Learning" published in arxiv, a widely recognized platform for sharing preprints and scientific articles. This article is currently undergoing a rigorous review process.

Thanks to the Novelis research team - including Jessica López Espejel, Mahaman Sanoussi Yahaya Alassan, El Mehdi Chouham, El Hassane Ettifouri, Walid Dahhane - for their know-how and expertise.


“Large Language Models (LLMs) have exhibited remarkable performance on various Natural Language Processing (NLP) tasks. However, there is a current hot debate regarding their reasoning capacity. In this paper, we examine the performance of GPT-3.5 and GPT-4 models, by performing a thorough technical evaluation on different reasoning tasks across eleven distinct datasets. Our findings show that GPT-4 outperforms GPT-3.5 in zero-shot learning throughout almost all evaluated tasks. In addition, we note that both models exhibit limited performance in Inductive, Mathematical, and Multi-hop Reasoning Tasks. While it may seem intuitive that the GPT-4 model would outperform GPT-3.5 given its size and efficiency in various NLP tasks, our paper offers empirical evidence to support this claim. We provide a detailed and comprehensive analysis of the results from both models to further support our findings. In addition, we propose a set of engineered prompts that improves performance of both models on zero-shot learning.”

arXiv is an open archive of electronic preprints of scientific articles in various technical fields, such as physics, mathematics, computer science and more, freely accessible via the Internet.

Cornell University is a private American university located primarily in the city of Ithaca, New York.

Recent blogs

Top 10 great language models that have transformed NLP in the last 5 years

GPT-4, released by OpenAI in 2023, is the language model that holds one of the largest neural netwo...

Novelis developed a ChatGPT connector for SS&C Blue Prism 

With the rapid advancement of technology, businesses are constantly striving to streamline their pr...

How to effectively manage your Service Level Agreement (SLA) with process intelligence?

Whether you are an operational center of excellence, a customer service department, or a Shared Ser...