{"id":8688,"date":"2023-05-23T14:49:29","date_gmt":"2023-05-23T14:49:29","guid":{"rendered":"https:\/\/novelis.io\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/"},"modified":"2024-04-19T13:53:28","modified_gmt":"2024-04-19T13:53:28","slug":"gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning","status":"publish","type":"scientific-pub","link":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/","title":{"rendered":"GPT-3.5 vs GPT-4: Evaluating ChatGPT’s Reasoning Performance in Zero-shot Learning"},"content":{"rendered":"\n

Discover the first version of our scientific publication „GPT-3.5 vs GPT-4: Evaluating ChatGPT’s Reasoning Performance in Zero-shot Learning“ published in arxiv<\/a>, a widely recognized platform for sharing preprints and scientific articles. This article is currently undergoing a rigorous review process.<\/p>\n\n\n\n

Thanks to the Novelis research team<\/a> – including Jessica L\u00f3pez Espejel, Mahaman Sanoussi Yahaya Alassan, El Mehdi Chouham, El Hassane Ettifouri, Walid Dahhane – for their know-how and expertise.<\/p>\n\n\n\n

\n
Read the full article<\/a><\/div>\n<\/div>\n\n\n\n
<\/div>\n\n\n\n

Abstract<\/h2>\n\n\n\n

\u201cLarge Language Models (LLMs) have exhibited remarkable performance on various Natural Language Processing (NLP) tasks. However, there is a current hot debate regarding their reasoning capacity. In this paper, we examine the performance of GPT-3.5 and GPT-4 models, by performing a thorough technical evaluation on different reasoning tasks across eleven distinct datasets. Our findings show that GPT-4 outperforms GPT-3.5 in zero-shot learning throughout almost all evaluated tasks. In addition, we note that both models exhibit limited performance in Inductive, Mathematical, and Multi-hop Reasoning Tasks. While it may seem intuitive that the GPT-4 model would outperform GPT-3.5 given its size and efficiency in various NLP tasks, our paper offers empirical evidence to support this claim. We provide a detailed and comprehensive analysis of the results from both models to further support our findings. In addition, we propose a set of engineered prompts that improves performance of both models on zero-shot learning.\u201d<\/p>\n\n\n\n

<\/div>\n\n\n\n

arXiv is an open archive of electronic preprints of scientific articles in various technical fields, such as physics, mathematics, computer science and more, freely accessible via the Internet.<\/strong><\/p>\n","protected":false},"featured_media":8333,"template":"","categories":[432],"custom_tag":[451,452,453],"class_list":["post-8688","scientific-pub","type-scientific-pub","status-publish","has-post-thumbnail","hentry","category-scientific-publication-de","custom_tag-chatgpt-de","custom_tag-gpt-3-5-de","custom_tag-gpt-4-de"],"acf":{"filter_opacity":"70","reading_time":"","authors":"","externel_link":"","summary":"","subtitle":"","document_to_download":{"upload_a_file":false,"download_without_form":false,"file":false,"url":""}},"yoast_head":"\nGPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance<\/title>\n<meta name=\"description\" content=\"Discover our scientific publication "GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance in Zero-shot Learning" published in arxiv.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance\" \/>\n<meta property=\"og:description\" content=\"Discover our scientific publication "GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance in Zero-shot Learning" published in arxiv.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/\" \/>\n<meta property=\"og:site_name\" content=\"Novelis innovation\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/novelis.io\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-19T13:53:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@novelis_io\" \/>\n<meta name=\"twitter:label1\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data1\" content=\"1\u00a0Minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/\",\"url\":\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/\",\"name\":\"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance\",\"isPartOf\":{\"@id\":\"https:\/\/novelis.io\/de\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg\",\"datePublished\":\"2023-05-23T14:49:29+00:00\",\"dateModified\":\"2024-04-19T13:53:28+00:00\",\"description\":\"Discover our scientific publication \\\"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance in Zero-shot Learning\\\" published in arxiv.\",\"breadcrumb\":{\"@id\":\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#primaryimage\",\"url\":\"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg\",\"contentUrl\":\"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg\",\"width\":1200,\"height\":628},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/novelis.io\/de\/?page_id=7469\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"GPT-3.5 vs GPT-4: Evaluating ChatGPT’s Reasoning Performance in Zero-shot Learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/novelis.io\/de\/#website\",\"url\":\"https:\/\/novelis.io\/de\/\",\"name\":\"Novelis innovation\",\"description\":\"Novelis innovation\",\"publisher\":{\"@id\":\"https:\/\/novelis.io\/de\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/novelis.io\/de\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/novelis.io\/de\/#organization\",\"name\":\"Novelis innovation\",\"url\":\"https:\/\/novelis.io\/de\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/novelis.io\/de\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png\",\"contentUrl\":\"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png\",\"width\":479,\"height\":98,\"caption\":\"Novelis innovation\"},\"image\":{\"@id\":\"https:\/\/novelis.io\/de\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/novelis.io\",\"https:\/\/x.com\/novelis_io\",\"https:\/\/www.linkedin.com\/company\/novelis-consulting\/\",\"https:\/\/www.youtube.com\/channel\/UCJ5eJR22n2GtfKaTWueWRPQ\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance","description":"Discover our scientific publication \"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance in Zero-shot Learning\" published in arxiv.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/","og_locale":"de_DE","og_type":"article","og_title":"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance","og_description":"Discover our scientific publication \"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance in Zero-shot Learning\" published in arxiv.","og_url":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/","og_site_name":"Novelis innovation","article_publisher":"https:\/\/www.facebook.com\/novelis.io","article_modified_time":"2024-04-19T13:53:28+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@novelis_io","twitter_misc":{"Gesch\u00e4tzte Lesezeit":"1\u00a0Minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/","url":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/","name":"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance","isPartOf":{"@id":"https:\/\/novelis.io\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#primaryimage"},"image":{"@id":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#primaryimage"},"thumbnailUrl":"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg","datePublished":"2023-05-23T14:49:29+00:00","dateModified":"2024-04-19T13:53:28+00:00","description":"Discover our scientific publication \"GPT-3.5 vs GPT-4: Evaluating ChatGPT's Reasoning Performance in Zero-shot Learning\" published in arxiv.","breadcrumb":{"@id":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#primaryimage","url":"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg","contentUrl":"https:\/\/novelis.io\/wp-content\/uploads\/2023\/05\/BLOG-1200-\u00d7-628-px-5.jpg","width":1200,"height":628},{"@type":"BreadcrumbList","@id":"https:\/\/novelis.io\/de\/scientific-pub\/gpt-3-5-vs-gpt-4-evaluating-chatgpts-reasoning-performance-in-zero-shot-learning\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/novelis.io\/de\/?page_id=7469"},{"@type":"ListItem","position":2,"name":"GPT-3.5 vs GPT-4: Evaluating ChatGPT’s Reasoning Performance in Zero-shot Learning"}]},{"@type":"WebSite","@id":"https:\/\/novelis.io\/de\/#website","url":"https:\/\/novelis.io\/de\/","name":"Novelis innovation","description":"Novelis innovation","publisher":{"@id":"https:\/\/novelis.io\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/novelis.io\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/novelis.io\/de\/#organization","name":"Novelis innovation","url":"https:\/\/novelis.io\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/novelis.io\/de\/#\/schema\/logo\/image\/","url":"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png","contentUrl":"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png","width":479,"height":98,"caption":"Novelis innovation"},"image":{"@id":"https:\/\/novelis.io\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/novelis.io","https:\/\/x.com\/novelis_io","https:\/\/www.linkedin.com\/company\/novelis-consulting\/","https:\/\/www.youtube.com\/channel\/UCJ5eJR22n2GtfKaTWueWRPQ"]}]}},"_links":{"self":[{"href":"https:\/\/novelis.io\/de\/wp-json\/wp\/v2\/scientific-pub\/8688"}],"collection":[{"href":"https:\/\/novelis.io\/de\/wp-json\/wp\/v2\/scientific-pub"}],"about":[{"href":"https:\/\/novelis.io\/de\/wp-json\/wp\/v2\/types\/scientific-pub"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/novelis.io\/de\/wp-json\/wp\/v2\/media\/8333"}],"wp:attachment":[{"href":"https:\/\/novelis.io\/de\/wp-json\/wp\/v2\/media?parent=8688"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/novelis.io\/de\/wp-json\/wp\/v2\/categories?post=8688"},{"taxonomy":"custom_tag","embeddable":true,"href":"https:\/\/novelis.io\/de\/wp-json\/wp\/v2\/custom_tag?post=8688"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}