Artificial intelligence (AI)-assisted programming can enable software engineers to work more efficiently and effectively with the existing software tools such as OpenAI ChatGPT, Github Copilot, DeepMind AlphaCode, Amazon Codewhisperer, Replit Ghostwriter, Microsoft IntelliCode and Codedium, especially in situations where complex algorithms are being used that involve large amounts of code (i.e., Big Code regime). It also strikes a balance between productivity and ensuring safety, security, and reliability within the programming development environment. There are two main categories of AI-assisted programming tasks related to software naturalness: generation and understanding. The former includes code generation, code completion, code translation, code refinement, and code summarization. The latter is concerned with understanding code and includes defect detection and clone detection.
Framework |
Year |
Task(s) |
Baseline(s) |
Supported Language(s) |
Open Sourced |
---|---|---|---|---|---|
Refactory [70] |
2019 |
Defect Detection |
BLEU |
Java |
✗ |
CuBERT [71] |
2020 |
Code Refinement, Defect Detection |
BERT |
Python |
✓ |
CugLM [72] |
2020 |
Code Completion |
BERT |
Java, TypeScript |
✓ |
Intellicode [73] |
2020 |
Code Generation, Code Completion |
GPT-2 |
Python, C#, JavaScript, and TypeScrip |
✗ |
Great [74] |
2020 |
Defect Detection |
Vanilla Transformers |
Python |
✓ |
TreeGEN [75] |
2020 |
Code Generation |
Vanilla Transformers |
Python |
✓ |
C-BERT [59] |
2020 |
Defect Detection |
BERT |
C |
✗ |
TransCoder [76] |
2020 |
Code Translation |
Vanilla Transformers |
C++, Java, and Python |
✗ |
GraphCodeBERT [77] |
2020 |
Code Summarization, Code Refinement |
BERT |
Java |
✗ |
Codex [9] |
2021 |
Code Generation, Code Completion, Code Summarization, Benchmark |
GPT-3 |
JavaScript, Go, Perl, and 6 more |
✗ |
Copilot [78] |
2021 |
Code Generation, Code Completion |
Codex |
Java, PHP, Python, and 5 more |
✗ |
CodeT5 [79] |
2021 |
Code Summarization, Code Generation, Code Translation, Code Refinement, Defect Detection, Clone Detection |
T5 |
Python, Java |
✓ |
Tfix [80] |
2021 |
Code Refinement, Defect Detection |
T5 |
JavaScript |
✓ |
CodeRL [81] |
2021 |
Code Summarization, Code Generation, Code Translation, Code Refinement, Defect Detection, Clone Detection |
T5 |
Java |
✓ |
TreeBERT [82] |
2021 |
Code Summarization |
Vanilla Transformers |
Python, Java |
✓ |
BUGLAB [83] |
2021 |
Code Refinement, Defect Detection |
GREAT |
Python |
✓ |
TBCC [84] |
2021 |
Clone Detection |
Vanilla Transformers |
C, Java |
✓ |
APPS [11] |
2021 |
Benchmark |
N/A |
Python |
✓ |
CodeXGLUE [19] |
2021 |
Benchmark |
N/A |
Python |
✓ |
CoTexT [85] |
2021 |
Code Summarization, Code Generation, Code Refinement, Defect detection |
T5 |
Python, Java, Javascript, PHP, Ruby, Go |
✓ |
SynCoBERT [86] |
2021 |
Code Translation, Defect Detection, Clone Detection |
BERT |
Ruby, Javascript, Go, Python, Java, PHP |
✗ |
TravTrans [87] |
2021 |
Code Completion |
Vanilla Transformers |
Python |
✗ |
CCAG [88] |
2021 |
Code Completion |
Vanilla Transformers |
JavaScript, Python |
✗ |
DeepDebug [89] |
2021 |
Defect Detection |
Reformer |
Java |
✓ |
Recoder [25] |
2021 |
Defect Detection |
TreeGen |
Java |
✓ |
PLBART [90] |
2021 |
Code Summarization, Code Generation, Code Translation, Code Refinement, Clone Detection, Detect Detection |
BART |
Java, Python |
✗ |
CODEGEN [91] |
2022 |
Code Generation |
GPT-NEO & GPT-J |
Python |
✓ |
GPT-2 for APR [92] |
2022 |
Code Refinement |
GPT-2 |
JavaScript |
✓ |
CERT [93] |
2022 |
Code Generation |
CODEGEN |
Python |
✓ |
PyCoder [18] |
2022 |
Code Generation |
GPT-2 |
Python |
✓ |
AlphaCode [94] |
2022 |
Code Generation |
GPT |
Java |
✗ |
InCoder [95] |
2022 |
Code Generation, Code Completion, Code Summarization |
GPT-3 |
Java, JavaScript, Python |
✓ |
RewardRepair [96] |
2022 |
Code Refinement, Defect Detection |
T5 |
Java |
✓ |
CodeParrot [97] |
2022 |
Code Generation |
GPT-2 |
Python |
✓ |
AlphaRepair [98] |
2022 |
Code Refinement, Defect Detection |
CodeBERT |
Java |
✓ |
CodeReviewer [60] |
2022 |
Code Summarization, Code Refinement, Defect Detection |
CodeT5 |
Java |
✓ |
TransRepair [99] |
2022 |
Code Refinement, Defect Detection |
BLEU |
Java |
✗ |
NatGen [100] |
2022 |
Code Generation, Code Translation, Code Refinement |
CodeT5 |
Java, Python, Go, JavaScript, Ruby, PHP |
✓ |
DualSC [101] |
2022 |
Code Generation, Code Summarization |
T5 |
Shellcode |
✓ |
VulRepair [102] |
2022 |
Code Refinement, Defect Detection |
T5 |
C, C++ |
✓ |
CoditT5 [103] |
2022 |
Code Summarization, Defect Detection |
CodeT5 |
Java, Python, Ruby, PHP, Go, JavaScript |
✓ |
C4 [104] |
2022 |
Clone Detection |
CodeBERT |
C++, C#, Java, Python |
✓ |
SPT-Code [105] |
2022 |
Code Summarization, Code Completion, Code Refinement, Code Translation |
CodeBERT & GraphCodeBERT |
Python, Java, JavaScript, PHP, Go |
✓ |
ExploitGen [106] |
2023 |
Code Generation |
CodeBERT |
Python, Assembly |
✓ |
Santacoder [107] |
2023 |
Code Summarization, Code Generation |
GPT-2 |
Python, Java, and Javascript |
✓ |
xCodeEval [108] |
2023 |
Benchmark |
N/A |
Python, Java, C++, PHP, and 8 more |
✓ |
StarCoder [109] |
2023 |
Code Generation, Code Completion, Code Summarization |
BERT & SantaCoder |
HTML, Python, Java, and 83 more |
✓ |
This entry is adapted from the peer-reviewed paper 10.3390/e25060888