Analysis of ChatGPT-Generated Codes Across Multiple Programming Languages
Our research focuses on the intersection of artificial intelligence (AI) and software development, particularly the role of AI models in automating code generation. With advancements in large language models like ChatGPT, developers can now generate code from natural language prompts, a task that tr...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10870152/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823859654793363456 |
|---|---|
| author | Sally Almanasra Khaled Suwais |
| author_facet | Sally Almanasra Khaled Suwais |
| author_sort | Sally Almanasra |
| collection | DOAJ |
| description | Our research focuses on the intersection of artificial intelligence (AI) and software development, particularly the role of AI models in automating code generation. With advancements in large language models like ChatGPT, developers can now generate code from natural language prompts, a task that traditionally required significant manual input and expertise. AI-generated code promises to boost productivity by enabling faster prototyping and automating repetitive coding tasks. However, as these models are increasingly adopted in real-world applications, questions surrounding their efficiency and code quality become critical. This research investigates ChatGPT-4o, a state-of-the-art language model, and its ability to generate functional, high-quality code in different programming languages. By comparing performance between Python and Java, the study seeks to shed light on AI’s capabilities and limitations in code generation, addressing not only functional correctness but also broader software engineering concerns such as memory usage, runtime efficiency, and maintainability. The study addresses key questions related to the performance, code quality, and error management of AI-generated code by analyzing solutions for 300 data structure problems and 300 problems from the LeetCode platform. The findings reveal notable performance differences between the two languages: Java demonstrated superior runtime performance, particularly for medium and hard problems, while Python exhibited better memory efficiency across all complexity levels. The research also highlighted significant gaps in code quality, with both languages showing deficiencies in documentation and exception management. This study contributes to the literature by offering a comprehensive cross-language analysis of ChatGPT-4o’s programming capabilities, addressing a gap in the evaluation of AI-generated code performance. |
| format | Article |
| id | doaj-art-f23495dfd70541069c638be761a2b00f |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-f23495dfd70541069c638be761a2b00f2025-02-11T00:01:10ZengIEEEIEEE Access2169-35362025-01-0113235802359610.1109/ACCESS.2025.353805010870152Analysis of ChatGPT-Generated Codes Across Multiple Programming LanguagesSally Almanasra0https://orcid.org/0000-0001-8681-1231Khaled Suwais1https://orcid.org/0000-0001-6530-5022Faculty of Computer Studies, Arab Open University, Riyadh, Saudi ArabiaFaculty of Computer Studies, Arab Open University, Riyadh, Saudi ArabiaOur research focuses on the intersection of artificial intelligence (AI) and software development, particularly the role of AI models in automating code generation. With advancements in large language models like ChatGPT, developers can now generate code from natural language prompts, a task that traditionally required significant manual input and expertise. AI-generated code promises to boost productivity by enabling faster prototyping and automating repetitive coding tasks. However, as these models are increasingly adopted in real-world applications, questions surrounding their efficiency and code quality become critical. This research investigates ChatGPT-4o, a state-of-the-art language model, and its ability to generate functional, high-quality code in different programming languages. By comparing performance between Python and Java, the study seeks to shed light on AI’s capabilities and limitations in code generation, addressing not only functional correctness but also broader software engineering concerns such as memory usage, runtime efficiency, and maintainability. The study addresses key questions related to the performance, code quality, and error management of AI-generated code by analyzing solutions for 300 data structure problems and 300 problems from the LeetCode platform. The findings reveal notable performance differences between the two languages: Java demonstrated superior runtime performance, particularly for medium and hard problems, while Python exhibited better memory efficiency across all complexity levels. The research also highlighted significant gaps in code quality, with both languages showing deficiencies in documentation and exception management. This study contributes to the literature by offering a comprehensive cross-language analysis of ChatGPT-4o’s programming capabilities, addressing a gap in the evaluation of AI-generated code performance.https://ieeexplore.ieee.org/document/10870152/Artificial intelligencesoftware developmentChatGPTlarge language modelprogramming |
| spellingShingle | Sally Almanasra Khaled Suwais Analysis of ChatGPT-Generated Codes Across Multiple Programming Languages IEEE Access Artificial intelligence software development ChatGPT large language model programming |
| title | Analysis of ChatGPT-Generated Codes Across Multiple Programming Languages |
| title_full | Analysis of ChatGPT-Generated Codes Across Multiple Programming Languages |
| title_fullStr | Analysis of ChatGPT-Generated Codes Across Multiple Programming Languages |
| title_full_unstemmed | Analysis of ChatGPT-Generated Codes Across Multiple Programming Languages |
| title_short | Analysis of ChatGPT-Generated Codes Across Multiple Programming Languages |
| title_sort | analysis of chatgpt generated codes across multiple programming languages |
| topic | Artificial intelligence software development ChatGPT large language model programming |
| url | https://ieeexplore.ieee.org/document/10870152/ |
| work_keys_str_mv | AT sallyalmanasra analysisofchatgptgeneratedcodesacrossmultipleprogramminglanguages AT khaledsuwais analysisofchatgptgeneratedcodesacrossmultipleprogramminglanguages |