Högskolan i Skövde

his.sePublications
System disruptions
We are currently experiencing disruptions on the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The impact of task specification on code generated via ChatGPT
University of Skövde, School of Informatics.
University of Skövde, School of Informatics.
University of Skövde, School of Informatics.
2023 (Swedish)Independent thesis Basic level (degree of Bachelor), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

ChatGPT has made large language models more accessible and made it possible to code using natural language prompts. This study conducted an experiment comparing prompt engineering techniques called task specification and investigated their impacton code generation in terms of correctness and variety. The hypotheses of this study focused on whether the baseline method had a statistically significant difference in code correctness compared to the other methods. Code is evaluated using a software requirement specification that measures functional and syntactical correctness. Additionally, code variance is measured to identify patterns in code generation. The results show that there is a statistically significant difference in some code correctness criteria between the baseline and the other task specification methods, and the code variance measurements indicate a variety in the generated solutions. Future work could include using another large language model; different programming tasks andprogramming languages; and other prompt engineering techniques.

Place, publisher, year, edition, pages
2023. , p. 60, 29
Keywords [en]
Code generation, task specification, prompt engineering, ChatGPT, human evaluation, code similarity
National Category
Information Systems, Social aspects
Identifiers
URN: urn:nbn:se:his:diva-22866OAI: oai:DiVA.org:his-22866DiVA, id: diva2:1776870
Subject / course
Informationsteknologi
Educational program
Computer Science - Specialization in Systems Development
Supervisors
Examiners
Available from: 2023-06-28 Created: 2023-06-28 Last updated: 2023-06-28Bibliographically approved

Open Access in DiVA

fulltext(3538 kB)773 downloads
File information
File name FULLTEXT01.pdfFile size 3538 kBChecksum SHA-512
77da424870a337c7b97dce6a325cb0d64799d6df1fa35e29adaa380733a2082eff2bb3964284c8557d55dd50b96cba22db35547d85650315efed5824cd2e7e3e
Type fulltextMimetype application/pdf

By organisation
School of Informatics
Information Systems, Social aspects

Search outside of DiVA

GoogleGoogle Scholar
Total: 773 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 830 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf