Découvrez notre publication scientifique « GPT-3.5, GPT-4, or BARD? Evaluating LLMs reasoning ability in zero-shot learning and performance boosting through prompts » publiée dans Elsevier et repris dans ScienceDirect. Cet article est en anglais.
Merci à l’équipe de recherche de Novelis – notamment Jessica López Espejel, Mahaman Sanoussi Yahaya Alassan, El Mehdi Chouham, El Hassane Ettifouri, Walid Dahhane – pour son savoir-faire et son expertise.
A propos
“Large Language Models (LLMs) have exhibited remarkable performance on various Natural Language Processing (NLP) tasks. However, there is a current hot debate regarding their reasoning capacity. In this paper, we examine the performance of GPT-3.5, GPT-4, and BARD models, by performing a thorough technical evaluation on different reasoning tasks across eleven distinct datasets. Our paper provides empirical evidence showcasing the superior performance of ChatGPT-4 in comparison to both ChatGPT-3.5 and BARD in zero-shot setting throughout almost all evaluated tasks. While the superiority of GPT-4 compared to GPT-3.5 might be explained by its larger size and NLP efficiency, this was not evident for BARD. We also demonstrate that the three models show limited proficiency in Inductive, Mathematical, and Multi-hop Reasoning Tasks. To bolster our findings, we present a detailed and comprehensive analysis of the results from these three models. Furthermore, we propose a set of engineered prompts that enhances the zero-shot setting performance of all three models.”
Elsevier est une entreprise d’analyse de données qui aide les institutions, les professionnels de santé et des sciences à améliorer leurs performances pour le bien-être de l’humanité.
ScienceDirect est la première source mondiale de recherche scientifique, technique et médicale.
Découvrez la première version de notre publication scientifique « GPT-3.5 vs GPT-4: Evaluating ChatGPT’s Reasoning Performance in Zero-shot Learning » publié dans arxiv, plateforme largement reconnue pour le partage de préprints et d’articles scientifiques. Cet article rédigé en anglais fait actuellement l’objet d’un processus de révision rigoureux.
Merci à l’équipe de recherche de Novelis – notamment Jessica López Espejel, Mahaman Sanoussi Yahaya Alassan, El Mehdi Chouham, El Hassane Ettifouri, Walid Dahhane – pour son savoir-faire et son expertise.
A propos
“Large Language Models (LLMs) have exhibited remarkable performance on various Natural Language Processing (NLP) tasks. However, there is a current hot debate regarding their reasoning capacity. In this paper, we examine the performance of GPT-3.5 and GPT-4 models, by performing a thorough technical evaluation on different reasoning tasks across eleven distinct datasets. Our findings show that GPT-4 outperforms GPT-3.5 in zero-shot learning throughout almost all evaluated tasks. In addition, we note that both models exhibit limited performance in Inductive, Mathematical, and Multi-hop Reasoning Tasks. While it may seem intuitive that the GPT-4 model would outperform GPT-3.5 given its size and efficiency in various NLP tasks, our paper offers empirical evidence to support this claim. We provide a detailed and comprehensive analysis of the results from both models to further support our findings. In addition, we propose a set of engineered prompts that improves performance of both models on zero-shot learning.”
arXiv est une archive ouverte de prépublications électroniques d’articles scientifiques dans différents domaines techniques, tels que la physique, les mathématiques, l’informatique et bien plus encore, gratuitement accessible par Internet.
L’université Cornell est une université privée américaine située principalement dans la ville d’Ithaca dans l’État de New York.
