{"id":10262,"date":"2024-04-19T14:03:20","date_gmt":"2024-04-19T14:03:20","guid":{"rendered":"https:\/\/novelis.io\/?post_type=scientific-pub&#038;p=10262"},"modified":"2024-04-22T09:46:35","modified_gmt":"2024-04-22T09:46:35","slug":"low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python","status":"publish","type":"scientific-pub","link":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/","title":{"rendered":"Mod\u00e8les de langage profonds low-cost : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python"},"content":{"rendered":"\n<p>D\u00e9couvrez la premi\u00e8re version de notre publication scientifique \u00ab\u00a0Mod\u00e8les de langage profonds low-cost : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python\u00a0\u00bb\u00a0publi\u00e9 dans <a href=\"https:\/\/arxiv.org\/abs\/2305.12477\" target=\"_blank\" rel=\"noreferrer noopener\">arxiv<\/a> et soumis au journal <strong>Engineering Applications of Artificial Intelligence.<\/strong> Cet article r\u00e9dig\u00e9 en anglais est d\u00e9j\u00e0 disponible au public.\u00a0<\/p>\n\n\n\n<p>Merci \u00e0 l\u2019<a href=\"https:\/\/novelis.io\/fr\/laboratoire-rd\/\">\u00e9quipe de recherche de Novelis<\/a>\u00a0&#8211; notamment Jessica\u00a0L\u00f3pez Espejel,\u00a0Mahaman Sanoussi\u00a0Yahaya Alassan,\u00a0Merieme Bouhandi,\u00a0Walid Dahhane, El Hassane\u00a0Ettifouri &#8211; pour son savoir-faire et son expertise.<\/p>\n\n\n\n<div style=\"height:100px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/arxiv.org\/abs\/2404.11160\" target=\"_blank\" rel=\"noopener\">Aller sur arxiv<\/a><\/div>\n<\/div>\n\n\n\n<div style=\"height:100px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"a-propos-de-l-etude\">A propos<\/h2>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>\u00ab\u00a0Large Language Models (LLMs) have become the go-to solution for many Natural Language Processing (NLP) tasks due to their ability to tackle various problems and produce high-quality results. Specifically, they are increasingly used to automatically generate code, easing the burden on developers by handling repetitive tasks. However, this improvement in quality has led to high computational and memory demands, making LLMs inaccessible to users with limited resources. In this paper, we focus on Central Processing Unit (CPU)-compatible models and conduct a thorough semi-manual evaluation of their strengths and weaknesses in generating Python code. We enhance their performance by introducing a Chain-of-Thought prompt that guides the model in problem-solving. Additionally, we propose a dataset of 60 programming problems with varying difficulty levels for evaluation purposes. Our assessment also includes testing these models on two state-of-the-art datasets: HumanEval and EvalPlus. We commit to sharing our dataset and experimental results publicly to ensure transparency.\u00a0\u00bb<\/p>\n\n\n\n<div style=\"height:100px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>arXiv est une archive ouverte de pr\u00e9publications \u00e9lectroniques d&rsquo;articles scientifiques dans diff\u00e9rents domaines techniques, tels que la physique, les math\u00e9matiques, l&rsquo;informatique et bien plus encore, gratuitement accessible par Internet.<\/strong><\/h4>\n<\/blockquote>\n","protected":false},"featured_media":10260,"template":"","categories":[24],"custom_tag":[80,460,495],"class_list":["post-10262","scientific-pub","type-scientific-pub","status-publish","has-post-thumbnail","hentry","category-publication-scientifique","custom_tag-code-fr","custom_tag-llm-fr","custom_tag-python-fr"],"acf":{"externel_link":"","summary":"","filter_opacity":"70","subtitle":"","reading_time":"","authors":"","document_to_download":{"upload_a_file":false,"download_without_form":true,"file":false,"url":"https:\/\/arxiv.org\/pdf\/2404.11160.pdf"},"show_recent_block_on_the_bottom_of_the_page":true},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.6 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Arxiv : Low-Cost Language Models - Novelis Research<\/title>\n<meta name=\"description\" content=\"Article scientifique &quot;Low-Cost Language Models : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python&quot;\u00a0publi\u00e9e dans\u00a0arxiv.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Arxiv : Low-Cost Language Models - Novelis Research\" \/>\n<meta property=\"og:description\" content=\"Article scientifique &quot;Low-Cost Language Models : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python&quot;\u00a0publi\u00e9e dans\u00a0arxiv.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/\" \/>\n<meta property=\"og:site_name\" content=\"Novelis innovation\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/novelis.io\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-22T09:46:35+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@novelis_io\" \/>\n<meta name=\"twitter:label1\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data1\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/\",\"url\":\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/\",\"name\":\"Arxiv : Low-Cost Language Models - Novelis Research\",\"isPartOf\":{\"@id\":\"https:\/\/novelis.io\/fr\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg\",\"datePublished\":\"2024-04-19T14:03:20+00:00\",\"dateModified\":\"2024-04-22T09:46:35+00:00\",\"description\":\"Article scientifique \\\"Low-Cost Language Models : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python\\\"\u00a0publi\u00e9e dans\u00a0arxiv.\",\"breadcrumb\":{\"@id\":\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#breadcrumb\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#primaryimage\",\"url\":\"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg\",\"contentUrl\":\"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg\",\"width\":1200,\"height\":628},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/novelis.io\/fr\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Mod\u00e8les de langage profonds low-cost : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/novelis.io\/fr\/#website\",\"url\":\"https:\/\/novelis.io\/fr\/\",\"name\":\"Novelis innovation\",\"description\":\"Novelis innovation\",\"publisher\":{\"@id\":\"https:\/\/novelis.io\/fr\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/novelis.io\/fr\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/novelis.io\/fr\/#organization\",\"name\":\"Novelis innovation\",\"url\":\"https:\/\/novelis.io\/fr\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/novelis.io\/fr\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png\",\"contentUrl\":\"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png\",\"width\":479,\"height\":98,\"caption\":\"Novelis innovation\"},\"image\":{\"@id\":\"https:\/\/novelis.io\/fr\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/novelis.io\",\"https:\/\/x.com\/novelis_io\",\"https:\/\/www.linkedin.com\/company\/novelis-consulting\/\",\"https:\/\/www.youtube.com\/channel\/UCJ5eJR22n2GtfKaTWueWRPQ\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Arxiv : Low-Cost Language Models - Novelis Research","description":"Article scientifique \"Low-Cost Language Models : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python\"\u00a0publi\u00e9e dans\u00a0arxiv.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/","og_locale":"fr_FR","og_type":"article","og_title":"Arxiv : Low-Cost Language Models - Novelis Research","og_description":"Article scientifique \"Low-Cost Language Models : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python\"\u00a0publi\u00e9e dans\u00a0arxiv.","og_url":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/","og_site_name":"Novelis innovation","article_publisher":"https:\/\/www.facebook.com\/novelis.io","article_modified_time":"2024-04-22T09:46:35+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@novelis_io","twitter_misc":{"Dur\u00e9e de lecture estim\u00e9e":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/","url":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/","name":"Arxiv : Low-Cost Language Models - Novelis Research","isPartOf":{"@id":"https:\/\/novelis.io\/fr\/#website"},"primaryImageOfPage":{"@id":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#primaryimage"},"image":{"@id":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#primaryimage"},"thumbnailUrl":"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg","datePublished":"2024-04-19T14:03:20+00:00","dateModified":"2024-04-22T09:46:35+00:00","description":"Article scientifique \"Low-Cost Language Models : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python\"\u00a0publi\u00e9e dans\u00a0arxiv.","breadcrumb":{"@id":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#breadcrumb"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/"]}]},{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#primaryimage","url":"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg","contentUrl":"https:\/\/novelis.io\/wp-content\/uploads\/2024\/04\/Low-Cost-Language-Models.jpg","width":1200,"height":628},{"@type":"BreadcrumbList","@id":"https:\/\/novelis.io\/fr\/scientific-pub\/low-cost-language-models-enquete-et-evaluation-des-performances-sur-la-generation-de-code-python\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/novelis.io\/fr\/"},{"@type":"ListItem","position":2,"name":"Mod\u00e8les de langage profonds low-cost : Enqu\u00eate et \u00e9valuation des performances sur la g\u00e9n\u00e9ration de code Python"}]},{"@type":"WebSite","@id":"https:\/\/novelis.io\/fr\/#website","url":"https:\/\/novelis.io\/fr\/","name":"Novelis innovation","description":"Novelis innovation","publisher":{"@id":"https:\/\/novelis.io\/fr\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/novelis.io\/fr\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"fr-FR"},{"@type":"Organization","@id":"https:\/\/novelis.io\/fr\/#organization","name":"Novelis innovation","url":"https:\/\/novelis.io\/fr\/","logo":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/novelis.io\/fr\/#\/schema\/logo\/image\/","url":"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png","contentUrl":"https:\/\/novelis.io\/wp-content\/uploads\/2021\/12\/logo-1.png","width":479,"height":98,"caption":"Novelis innovation"},"image":{"@id":"https:\/\/novelis.io\/fr\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/novelis.io","https:\/\/x.com\/novelis_io","https:\/\/www.linkedin.com\/company\/novelis-consulting\/","https:\/\/www.youtube.com\/channel\/UCJ5eJR22n2GtfKaTWueWRPQ"]}]}},"_links":{"self":[{"href":"https:\/\/novelis.io\/fr\/wp-json\/wp\/v2\/scientific-pub\/10262"}],"collection":[{"href":"https:\/\/novelis.io\/fr\/wp-json\/wp\/v2\/scientific-pub"}],"about":[{"href":"https:\/\/novelis.io\/fr\/wp-json\/wp\/v2\/types\/scientific-pub"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/novelis.io\/fr\/wp-json\/wp\/v2\/media\/10260"}],"wp:attachment":[{"href":"https:\/\/novelis.io\/fr\/wp-json\/wp\/v2\/media?parent=10262"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/novelis.io\/fr\/wp-json\/wp\/v2\/categories?post=10262"},{"taxonomy":"custom_tag","embeddable":true,"href":"https:\/\/novelis.io\/fr\/wp-json\/wp\/v2\/custom_tag?post=10262"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}