Awesome Code LLM

Code-llm ที่ยอดเยี่ยม

นี่คือ repo สำหรับการสำรวจ TMLR ของเรารวมมุมมองของ NLP และวิศวกรรมซอฟต์แวร์: การสำรวจรูปแบบภาษาสำหรับรหัส - การทบทวนที่ครอบคลุมของการวิจัย LLM สำหรับรหัส ผลงานในแต่ละหมวดหมู่ได้รับคำสั่งตามลำดับเวลา หากคุณมีความเข้าใจขั้นพื้นฐานเกี่ยวกับการเรียนรู้ของเครื่อง แต่ยังใหม่กับ NLP เรายังให้รายการการอ่านที่แนะนำในส่วนที่ 9

ข่าว

[2024/11/28] เอกสารเด่น:

• การเพิ่มประสิทธิภาพการตั้งค่าสำหรับการให้เหตุผลกับคำติชมหลอกจากมหาวิทยาลัยเทคโนโลยีนันง

• Scribeagent: ไปยังตัวแทนเว็บเฉพาะทางที่ใช้ข้อมูลเวิร์กโฟลว์ระดับการผลิตจาก Scribe

• การเขียนโปรแกรมที่ขับเคลื่อนด้วยการวางแผน: เวิร์กโฟลว์การเขียนโปรแกรมแบบจำลองภาษาขนาดใหญ่จากมหาวิทยาลัยเมลเบิร์น

• เกณฑ์มาตรฐานการแปลระดับที่เก็บของการกำหนดเป้าหมายสนิมจากมหาวิทยาลัยซุนยัตเซ็น

• การใช้ประโยชน์จากประสบการณ์ก่อนหน้านี้: ฐานความรู้เสริมที่ขยายได้สำหรับข้อความถึง SQL จากมหาวิทยาลัยวิทยาศาสตร์และเทคโนโลยีของจีน

• CodexEmbed: ตระกูล Embedding Generalist สำหรับการดึงรหัสหลายภาษาและหลายงานจากการวิจัย Salesforce AI

• POSEC: Fortifying Code LLMS ด้วยการจัดตำแหน่งความปลอดภัยเชิงรุกจาก Purdue University

[2024/10/22] เราได้รวบรวมเอกสาร 70 ฉบับตั้งแต่เดือนกันยายนและตุลาคม 2567 ในบทความ WeChat หนึ่งบทความ

[2024/09/06] การสำรวจของเราได้รับการยอมรับสำหรับการตีพิมพ์โดยการทำธุรกรรมเกี่ยวกับการวิจัยการเรียนรู้ของเครื่อง (TMLR)

[2024/09/14] เราได้รวบรวมเอกสาร 57 ฉบับตั้งแต่เดือนสิงหาคม 2567 (รวมถึง 48 ที่ ACL 2024) ในบทความ WeChat หนึ่งบทความ

วิธีการมีส่วนร่วม

หากคุณพบว่ากระดาษหายไปจากที่เก็บนี้วางผิดพลาดในหมวดหมู่หรือขาดการอ้างอิงถึงข้อมูลวารสาร/การประชุมโปรดอย่าลังเลที่จะสร้างปัญหา หากคุณพบว่า repo นี้มีประโยชน์โปรดอ้างอิงการสำรวจของเรา:

 @article{zhang2024unifying,
   title={Unifying the Perspectives of {NLP} and Software Engineering: A Survey on Language Models for Code},
   author={Ziyin Zhang and Chaoyu Chen and Bingchang Liu and Cong Liao and Zi Gong and Hang Yu and Jianguo Li and Rui Wang},
   journal={Transactions on Machine Learning Research},
   issn={2835-8856},
   year={2024},
   url={https://openreview.net/forum?id=hkNnGqZnpa},
   note={}
}

สารบัญ

1. การสำรวจ

2. แบบจำลอง

   2.1 ฐาน LLMS และกลยุทธ์การเตรียมการก่อน

   2.2 LLM ที่มีอยู่ปรับให้เข้ากับรหัส

   2.3 การเตรียมการทั่วไปในรหัส

   • เครื่องเข้ารหัส
   • ตัวถอดรหัส
   • เครื่องเข้ารหัส
   • unilm

   2.4 (คำแนะนำ) การปรับแต่งในรหัส

   2.5 การเรียนรู้การเสริมแรงทางรหัส

3. เมื่อการเขียนโค้ดตรงตามการใช้เหตุผล

   3.1 การเข้ารหัสเพื่อการให้เหตุผล

   3.2 การจำลองรหัส

   3.3 CODE AGENTS

   3.4 การเข้ารหัสแบบโต้ตอบ

   3.5 การนำทางส่วนหน้า

4. รหัส LLM สำหรับภาษาที่มีทรัพยากรต่ำระดับต่ำและภาษาเฉพาะโดเมน

5. วิธีการ/รุ่นสำหรับงานดาวน์สตรีม

   • การเขียนโปรแกรม

     • การสร้างรหัส
     • รหัสผ้าขี้ริ้ว
     • การจัดอันดับรหัส
     • การแปลรหัส
     • การแสดงความคิดเห็นและสรุปรหัส
     • การซ่อมแซมโปรแกรม
     • รหัสที่คล้ายคลึงกันและการฝัง (การตรวจจับโคลน, การค้นหารหัส)
     • การปรับโครงสร้างรหัส
     • ประเภทการทำนาย
     • การเข้ารหัสระดับที่เก็บ
     • การพัฒนาส่วนหน้า
     • ข้อความถึง SQL
     • หลักฐานโปรแกรม

   • การทดสอบและการปรับใช้

     • การสร้างแบบทดสอบ
     • รุ่น Oracle
     • การทดสอบการกลายพันธุ์
     • การทดสอบ Fuzz
     • การตรวจหาช่องโหว่
     • การตรวจจับรหัสที่เป็นอันตราย
     • การเพิ่มประสิทธิภาพคอมไพเลอร์
     • การวิเคราะห์แบบไบนารีและการสลายตัว

   • Devops

     • สร้างการสร้างข้อความ
     • รีวิวรหัส
     • การวิเคราะห์บันทึก
     • การกำหนดค่าซอฟต์แวร์

   • ความต้องการ

     • การสร้างแบบจำลองซอฟต์แวร์
     • วิศวกรรมความต้องการ

6. การวิเคราะห์รหัส Ai-Generated

   • ความปลอดภัยและช่องโหว่
   • ความถูกต้อง
   • ภาพหลอน
   • ประสิทธิภาพ
   • ความทนทาน
   • การตีความได้
   • การใช้ API
   • ความเป็นส่วนตัว
   • อคติ
   • การตรวจจับรหัส AI
   • คนอื่น

7. การมีปฏิสัมพันธ์ของมนุษย์-LLM

8. ชุดข้อมูล

   8.1 pretraining

   8.2 มาตรฐาน

   • เกณฑ์มาตรฐานแบบบูรณาการ
   • การสังเคราะห์โปรแกรม
   • การสังเคราะห์โปรแกรมที่มีพื้นฐานทางสายตา
   • การให้เหตุผลรหัสและ QA
   • ข้อความถึง SQL
   • การแปลรหัส
   • การซ่อมแซมโปรแกรม
   • การสรุปรหัส
   • การตรวจหาข้อบกพร่อง/ช่องโหว่
   • การดึงรหัส
   • พิมพ์การอนุมาน
   • สร้างการสร้างข้อความ
   • การเข้ารหัสระดับ repo

9. การอ่านที่แนะนำ

10. การอ้างอิง

11. ประวัติดาว

12. เข้าร่วมกับเรา

1. แบบสำรวจ

เราแสดงรายการแบบสำรวจล่าสุดหลายเรื่องในหัวข้อที่คล้ายกัน ในขณะที่พวกเขาทั้งหมดเกี่ยวกับรูปแบบภาษาสำหรับรหัส 1-2 มุ่งเน้นไปที่ด้าน NLP; 3-6 มุ่งเน้นไปที่ด้าน SE; 7-11 ได้รับการปล่อยตัวหลังจากเรา

1. "โมเดลภาษาขนาดใหญ่ตรงกับ NL2Code: การสำรวจ" [2022-12] [ACL 2023] [กระดาษ]

2. "การสำรวจแบบจำลองภาษาที่ผ่านการฝึกอบรมสำหรับสติปัญญารหัสประสาท" [2022-12] [กระดาษ]

3. "การเปรียบเทียบเชิงประจักษ์ของโมเดลที่ได้รับการฝึกอบรมมาก่อนของซอร์สโค้ด" [2023-02] [ICSE 2023] [กระดาษ]

4. "แบบจำลองภาษาขนาดใหญ่สำหรับวิศวกรรมซอฟต์แวร์: การทบทวนวรรณกรรมอย่างเป็นระบบ" [2023-08] [กระดาษ]

5. "สู่ความเข้าใจในรูปแบบภาษาขนาดใหญ่ในงานวิศวกรรมซอฟต์แวร์" [2023-08] [กระดาษ]

6. "ข้อผิดพลาดในรูปแบบภาษาสำหรับรหัสข่าวกรอง: อนุกรมวิธานและการสำรวจ" [2023-10] [กระดาษ]

7. "การสำรวจแบบจำลองภาษาขนาดใหญ่สำหรับวิศวกรรมซอฟต์แวร์" [2023-12] [กระดาษ]

8. "การเรียนรู้อย่างลึกซึ้งสำหรับรหัสข่าวกรอง: การสำรวจเกณฑ์มาตรฐานและชุดเครื่องมือ" [2023-12] [กระดาษ]

9. "การสำรวจความฉลาดของรหัสประสาท: กระบวนทัศน์ความก้าวหน้าและนอกเหนือจาก" [2024-03] [กระดาษ]

10. "งานที่ผู้คนแจ้ง: อนุกรมวิธานของงาน LLM ดาวน์สตรีมในการตรวจสอบซอฟต์แวร์และวิธีการปลอมแปลง" [2024-04] [กระดาษ]

11. "การเขียนโปรแกรมอัตโนมัติ: โมเดลภาษาขนาดใหญ่และเกินกว่า" [2024-05] [กระดาษ]

12. "โมเดลวิศวกรรมซอฟต์แวร์และรากฐาน: ข้อมูลเชิงลึกจากบล็อกอุตสาหกรรมโดยใช้คณะลูกขุนของโมเดลมูลนิธิ" [2024-10] [กระดาษ]

13. "วิศวกรรมซอฟต์แวร์ที่ใช้การเรียนรู้ลึก: ความคืบหน้าความท้าทายและโอกาส" [2024-10] [กระดาษ]

2. รุ่น

2.1 ฐาน LLMS และกลยุทธ์การเตรียมการก่อน

LLM เหล่านี้ไม่ได้รับการฝึกฝนเฉพาะสำหรับรหัส แต่ได้แสดงให้เห็นถึงความสามารถในการเข้ารหัสที่แตกต่างกัน

1. Lamda : "Lamda: รุ่นภาษาสำหรับแอปพลิเคชันโต้ตอบ" [2022-01] [กระดาษ]

2. ปาล์ม : "ปาล์ม: การสร้างแบบจำลองภาษาด้วยเส้นทาง" [2022-04] [JMLR] [กระดาษ]

3. GPT-NEOX : "GPT-NEOX-20B: โมเดลภาษาแบบอัตโนมัติโอเพนซอร์ซ" [2022-04] [ACL 2022 เวิร์กช็อปเกี่ยวกับความท้าทายและมุมมองในการสร้าง LLMS] [กระดาษ] [repo]

4. Bloom : "Bloom: รุ่นภาษาหลายภาษาแบบเปิดกว้าง 176b-parameter" [2022-11] [กระดาษ] [รุ่น]

5. LLAMA : "Llama: แบบเปิดและเปิดกว้างของรูปแบบภาษาพื้นฐาน" [2023-02] [กระดาษ]

6. GPT-4 : "รายงานทางเทคนิค GPT-4" [2023-03] [กระดาษ]

7. Llama 2 : "Llama 2: Open Foundation และ Models Chat ที่ปรับแต่งได้อย่างละเอียด" [2023-07] [กระดาษ] [repo]

8. Phi-1.5 : "ตำราเรียนเป็นสิ่งที่คุณต้องการ II: รายงานทางเทคนิค Phi-1.5" [2023-09] [กระดาษ] [รุ่น]

9. Baichuan 2 : "Baichuan 2: เปิดรุ่นภาษาขนาดใหญ่" [2023-09] [กระดาษ] [repo]

10. Qwen : "รายงานทางเทคนิค Qwen" [2023-09] [กระดาษ] [repo]

11. Mistral : "Mistral 7b" [2023-10] [กระดาษ] [repo]

12. ราศีเมถุน : "ราศีเมถุน: ตระกูลที่มีความสามารถสูงหลายรุ่น" [2023-12] [กระดาษ]

13. Phi-2 : "Phi-2: พลังที่น่าประหลาดใจของรุ่นภาษาขนาดเล็ก" [2023-12] [บล็อก]

14. Yayi2 : "Yayi 2: Multilingual Open-Source Language Models" [2023-12] [กระดาษ] [repo]

15. Deepseek : "Deepseek LLM: ปรับขนาดแบบจำลองภาษาโอเพ่นซอร์สด้วย longtermism" [2024-01] [กระดาษ] [repo]

16. Mixtral : "Mixtral of Experts" [2024-01] [Paper] [Blog]

17. Deepseekmoe : "DeepseekMoe: สู่ความเชี่ยวชาญเฉพาะด้านของผู้เชี่ยวชาญในรูปแบบภาษาผสมของ Experts" [2024-01] [กระดาษ] [repo]

18. Orion : "Orion-14b: Open-Source Multilingual Language Models" [2024-01] [กระดาษ] [repo]

19. Olmo : "Olmo: เร่งวิทยาศาสตร์แบบจำลองภาษา" [2024-02] [กระดาษ] [repo]

20. Gemma : "Gemma: แบบเปิดตามการวิจัยและเทคโนโลยีของราศีเมถุน" [2024-02] [กระดาษ] [บล็อก]

21. Claude 3 : "The Claude 3 Model Family: Opus, Sonnet, Haiku" [2024-03] [กระดาษ] [บล็อก]

22. Yi : "Yi: Open Foundation Model โดย 01.ai" [2024-03] [กระดาษ] [repo]

23. Poro : "Poro 34b และพรแห่งการพูดได้หลายภาษา" [2024-04] [กระดาษ] [รุ่น]

24. Jetmoe : "Jetmoe: ถึงประสิทธิภาพของ Llama2 ด้วย 0.1m ดอลลาร์" [2024-04] [กระดาษ] [repo]

25. ลามะ 3 : "The Llama 3 Herd of Models" [2024-04] [บล็อก] [repo] [กระดาษ]

26. Reka Core : "Reka Core, Flash และ Edge: ชุดของรุ่นภาษาหลายรูปแบบที่ทรงพลัง" [2024-04] [กระดาษ]

27. Phi-3 : "รายงานทางเทคนิค Phi-3: รูปแบบภาษาที่มีความสามารถสูงในโทรศัพท์ของคุณ" [2024-04] [กระดาษ]

28. OpenElm : "OpenElm: ครอบครัวโมเดลภาษาที่มีประสิทธิภาพพร้อมการฝึกอบรมโอเพนซอร์ซและกรอบการอนุมาน" [2024-04] [กระดาษ] [repo]

29. Tele-FLM : "รายงานทางเทคนิค Tele-FLM" [2024-04] [กระดาษ] [รุ่น]

30. Deepseek-V2 : "Deepseek-V2: โมเดลภาษาที่แข็งแกร่งประหยัดและมีประสิทธิภาพและมีประสิทธิภาพ" [2024-05] [กระดาษ] [repo]

31. Gecko : "Gecko: รูปแบบภาษากำเนิดสำหรับภาษาอังกฤษ, รหัสและเกาหลี" [2024-05] [กระดาษ] [รุ่น]

32. MAP-NEO : "MAP-NEO: ซีรี่ส์ภาษาขนาดใหญ่ที่มีความสามารถและโปร่งใสสูงและโปร่งใส" [2024-05] [กระดาษ] [repo]

33. Skywork-Moe : "Skywork-Moe: การดำน้ำลึกลงไปในเทคนิคการฝึกอบรมสำหรับแบบจำลองภาษาผสมของ Experts" [2024-06] [กระดาษ]

34. XMODEL-LM : "รายงานทางเทคนิค XMODEL-LM" [2024-06] [กระดาษ]

35. GEB : "GEB-1.3B: เปิดรุ่นภาษาขนาดใหญ่ที่มีน้ำหนักเบา" [2024-06] [กระดาษ]

36. กระต่าย : "กระต่าย: นักบวชมนุษย์กุญแจสู่ประสิทธิภาพของแบบจำลองภาษาขนาดเล็ก" [2024-06] [กระดาษ]

37. DCLM : "DataComp-LM: ในการค้นหาชุดการฝึกอบรมรุ่นต่อไปสำหรับรูปแบบภาษา" [2024-06] [กระดาษ]

38. Nemotron-4 : "Nemotron-4 340b รายงานทางเทคนิค" [2024-06] [กระดาษ]

39. chatglm : "chatglm: ครอบครัวของรุ่นภาษาขนาดใหญ่จาก GLM-130B ถึง GLM-4 เครื่องมือทั้งหมด" [2024-06] [กระดาษ]

40. Yulan : "Yulan: รูปแบบภาษาขนาดใหญ่โอเพนซอร์ซ" [2024-06] [กระดาษ]

41. Gemma 2 : "Gemma 2: การปรับปรุงโมเดลภาษาแบบเปิดในขนาดที่ใช้งานได้จริง" [2024-06] [กระดาษ]

42. H2O-Danube3 : "รายงานทางเทคนิค H2O-Danube3" [2024-07] [กระดาษ]

43. QWEN2 : "รายงานทางเทคนิค QWEN2" [2024-07] [กระดาษ]

44. Allam : "Allam: โมเดลภาษาขนาดใหญ่สำหรับภาษาอาหรับและภาษาอังกฤษ" [2024-07] [กระดาษ]

45. SEALLMS 3 : "SEALLMS 3: Foundation Open และ Chat Multilingual Language Models สำหรับภาษาเอเชียตะวันออกเฉียงใต้" [2024-07] [กระดาษ]

46. AFM : "โมเดลภาษา Apple Intelligence Foundation" [2024-07] [กระดาษ]

47. "ถึงรหัสหรือไม่เป็นรหัส? การสำรวจผลกระทบของรหัสในการฝึกอบรมล่วงหน้า" [2024-08] [กระดาษ]

48. Olmoe : "Olmoe: เปิดแบบจำลองภาษาผสมของ Experts" [2024-09] [กระดาษ]

49. "การฝึกฝนรหัสมีผลต่อประสิทธิภาพการทำงานของแบบจำลองภาษาอย่างไร" [2024-09] [กระดาษ]

50. EUROLLM : "EUROLLM: แบบจำลองภาษาหลายภาษาสำหรับยุโรป" [2024-09] [กระดาษ]

51. "ภาษาการเขียนโปรแกรมใดและคุณสมบัติใดในขั้นตอนการฝึกอบรมก่อนส่งผลกระทบต่อประสิทธิภาพการอนุมานเชิงตรรกะแบบปลายน้ำ" [2024-10] [กระดาษ]

52. GPT-4O : "การ์ดระบบ GPT-4O" [2024-10] [กระดาษ]

53. Hunyuan-Large : "Hunyuan-Large: โมเดลโมโอโอเพนซอร์สที่มีพารามิเตอร์เปิดใช้งาน 52 พันล้านโดย Tencent" [2024-11] [กระดาษ]

54. Crystal : "Crystal: การส่องสว่างความสามารถของ LLM ในภาษาและรหัส" [2024-11] [กระดาษ]

55. XMODEL-1.5 : "XMODEL-1.5: LLM หลายภาษา 1B" [2024-11] [กระดาษ]

2.2 LLM ที่มีอยู่ปรับให้เข้ากับรหัส

โมเดลเหล่านี้เป็น LLM ที่มีวัตถุประสงค์ทั่วไปเพิ่มเติมเกี่ยวกับข้อมูลที่เกี่ยวข้องกับรหัส

• Codex (GPT-3): "การประเมินรูปแบบภาษาขนาดใหญ่ที่ผ่านการฝึกอบรมเกี่ยวกับรหัส" [2021-07] [กระดาษ]

• ปาล์มโคเดอร์ (ปาล์ม): "ปาล์ม: การสร้างแบบจำลองภาษาด้วยทางเดิน" [2022-04] [JMLR] [กระดาษ]

• Minerva (Palm): "การแก้ปัญหาการใช้เหตุผลเชิงปริมาณด้วยโมเดลภาษา" [2022-06] [กระดาษ]

• ปาล์ม 2 * (ปาล์ม 2): "รายงานทางเทคนิค Palm 2" [2023-05] [กระดาษ]

• รหัส LLAMA (LLAMA 2): "Code Llama: Open Foundation Models สำหรับรหัส" [2023-08] [กระดาษ] [repo]

• LeMur (Llama 2): "LeMur: Harmonizing ภาษาธรรมชาติและรหัสสำหรับตัวแทนภาษา" [2023-10] [ICLR 2024 Spotlight] [Paper]

• BTX (LLAMA 2): "Branch-Train-Mix: Mixing Expert LLMS เป็นส่วนผสมของ Experts LLM" [2024-03] [กระดาษ]

• Hirope : "Hirope: การคาดการณ์ความยาวสำหรับโมเดลรหัสโดยใช้ตำแหน่งลำดับชั้น" [2024-03] [ACL 2024] [กระดาษ]

• "การเรียนรู้ข้อความรหัสและคณิตศาสตร์พร้อมกันผ่านการหลอมรวมรุ่นภาษาที่มีความเชี่ยวชาญสูง" [2024-03] [กระดาษ]

• CodeGemma : "CodeGemma: แบบเปิดรหัสตาม Gemma" [2024-04] [Paper] [Model]

• Deepseek-Coder-V2 : "DEEPSEEK-CODER-V2: การทำลายอุปสรรคของโมเดลซอร์ซแบบปิดในรหัสข่าวกรอง" [2024-06] [กระดาษ]

• "สัญญาและอันตรายของแบบจำลองการสร้างรหัสการทำงานร่วมกัน: ประสิทธิภาพและประสิทธิภาพการท่องจำ" [2024-09] [กระดาษ]

• QWEN2.5-CODER : "รายงานทางเทคนิค QWEN2.5-CODER" [2024-09] [กระดาษ]

• Lingma SWE-GPT : "Lingma SWE-GPT: รูปแบบภาษาการพัฒนาแบบเปิดเป็นศูนย์กลางสำหรับการปรับปรุงซอฟต์แวร์อัตโนมัติ" [2024-11] [กระดาษ]

2.3 การเตรียมการทั่วไปในรหัส

โมเดลเหล่านี้คือตัวเข้ารหัสหม้อแปลงตัวถอดรหัสและตัวพิมพ์ใหญ่ที่ได้รับการปรับแต่งจากศูนย์โดยใช้วัตถุประสงค์ที่มีอยู่สำหรับการสร้างแบบจำลองภาษาทั่วไป

เครื่องเข้ารหัส

1. Cubert (MLM + NSP): "การเรียนรู้และประเมินผลการฝังบริบทของซอร์สโค้ด" [2019-12] [ICML 2020] [กระดาษ] [repo]

2. Codebert (MLM + RTD): "Codebert: แบบจำลองที่ผ่านการฝึกอบรมมาก่อนสำหรับการเขียนโปรแกรมและภาษาธรรมชาติ" [2020-02] [EMNLP 2020 การค้นพบ] [กระดาษ] [repo]

3. GraphCodebert (MLM + DFG Edge Prediction + DFG การจัดตำแหน่งโหนด): "GraphCodebert: การแสดงรหัสก่อนการฝึกอบรมด้วยการไหลของข้อมูล" [2020-09] [ICLR 2021] [กระดาษ] [repo]

4. Syncobert (การทำนาย MLM + ตัวระบุ + การทำนาย Edge AST + การเรียนรู้แบบตรงกันข้าม): "Syncobert: การฝึกอบรมหลายแบบหลายโมดอลที่มีแนวทางสำหรับการเป็นตัวแทนรหัส" [2021-08] [กระดาษ]

5. Disco (MLM + ประเภทโหนด MLM + การเรียนรู้แบบตรงกันข้าม): "สู่การเรียนรู้ (DIS)-ความคล้ายคลึงกันของซอร์สโค้ดจากความแตกต่างของโปรแกรม" [2021-10] [ACL 2022] [กระดาษ]

6. Code-MVP (MLM + ประเภทการอนุมาน + การเรียนรู้แบบตรงกันข้าม): "Code-MVP: การเรียนรู้ที่จะเป็นตัวแทนซอร์สโค้ดจากหลายมุมมองด้วยการฝึกอบรมก่อนหน้า" [2022-05] [NAACL 2022 ทางเทคนิคแทร็ก] [กระดาษ]

7. รหัส (MLM + deobfuscation + การเรียนรู้แบบตัดกัน): "การเรียนรู้รหัสการเรียนรู้ในระดับ" [2024-02] [ICLR 2024] [กระดาษ]

8. Colsbert (MLM): "การปรับขนาดกฎหมายที่อยู่เบื้องหลังรูปแบบการทำความเข้าใจรหัส" [2024-02] [กระดาษ]

ตัวถอดรหัส

1. GPT-C (CLM): "IntelliCode Compose: การสร้างรหัสโดยใช้หม้อแปลง" [2020-05] [ESEC/FSE 2020] [กระดาษ]

2. CodeGpt (CLM): "CodexGlue: ชุดข้อมูลมาตรฐานการเรียนรู้ของเครื่องสำหรับการทำความเข้าใจรหัสและการสร้าง" [2021-02] [ชุดข้อมูล Neurips และมาตรฐาน 2021] [กระดาษ] [repo]

3. CodeParrot (CLM) [2021-12] [บล็อก]

4. Polycoder (CLM): "การประเมินอย่างเป็นระบบของแบบจำลองภาษาขนาดใหญ่ของรหัส" [2022-02] [DL4C@ICLR 2022] [กระดาษ] [repo]

5. codegen (CLM): "codegen: โมเดลภาษาขนาดใหญ่เปิดสำหรับรหัสที่มีการสังเคราะห์โปรแกรมหลายครั้ง" [2022-03] [ICLR 2023] [กระดาษ] [repo]

6. Incoder (การปิดบังเชิงสาเหตุ): "Incoder: แบบจำลองการกำเนิดสำหรับรหัสที่ infilling และการสังเคราะห์" [2022-04] [ICLR 2023] [กระดาษ] [repo]

7. PyCodegpt (CLM): "ใบรับรอง: การฝึกอบรมก่อนการสร้างภาพร่างสำหรับการสร้างรหัสที่เน้นห้องสมุด" [2022-06] [IJCAI-ECAI 2022] [กระดาษ] [repo]

8. Pangu-Coder (CLM): "Pangu-Coder: การสังเคราะห์โปรแกรมด้วยการสร้างแบบจำลองภาษาระดับฟังก์ชั่น" [2022-07] [กระดาษ]

9. Santacoder (FIM): "Santacoder: อย่าไปถึงดวงดาว!" [2023-01] [กระดาษ] [รุ่น]

10. Codegeex (CLM): "codegeex: โมเดลที่ผ่านการฝึกอบรมมาก่อนสำหรับการสร้างรหัสที่มีการประเมินหลายภาษาบน Humaneval-X" [2023-03] [กระดาษ] [repo]

11. StarCoder (FIM): "StarCoder: ขอให้แหล่งข่าวอยู่กับคุณ!" [2023-05] [กระดาษ] [รุ่น]

12. Phi-1 (CLM): "ตำราเรียนเป็นสิ่งที่คุณต้องการ" [2023-06] [กระดาษ] [รุ่น]

13. CodeFuse (CLM): "CodeFUSE-13B: รหัสภาษาหลายภาษาแบบหลายภาษาแบบจำลอง" [2023-10] [กระดาษ] [รุ่น]

14. Deepseek Coder (CLM+FIM): "Deepseek-Coder: เมื่อโมเดลภาษาขนาดใหญ่ตรงกับการเขียนโปรแกรม-การเพิ่มขึ้นของรหัสข่าวกรอง" [2024-01] [กระดาษ] [repo]

15. StarCoder2 (CLM+FIM): "StarCoder 2 และ Stack V2: The Next Generation" [2024-02] [Paper] [repo]

16. Codeshell (CLM+FIM): "รายงานทางเทคนิค Codeshell" [2024-03] [กระดาษ] [repo]

17. codeqwen1.5 [2024-04] [บล็อก]

18. หินแกรนิต : "โมเดลรหัสหินแกรนิต: ตระกูลของแบบจำลองฐานรากแบบเปิดสำหรับรหัสข่าวกรอง" [2024-05] [กระดาษ] "โมเดลรหัสหินแกรนิตเป็นบริบท 128K" [2024-07] [กระดาษ]

19. nt-java : "หม้อแปลงแคบ: Java-LM ที่ใช้ Starcoder สำหรับเดสก์ท็อป" [2024-07] [กระดาษ]

20. Arctic-SnowCoder : "Arctic-SnowCoder: ข้อมูลคุณภาพสูงในการฝึกฝนรหัส" [2024-09] [กระดาษ]

21. AixCoder : "AixCoder-7B: โมเดลภาษาขนาดใหญ่ที่มีน้ำหนักเบาและมีประสิทธิภาพสำหรับการกรอกรหัส" [2024-10] [กระดาษ]

22. OpenCoder : "OpenCoder: Open Cookbook สำหรับรหัสชั้นบนสุด ๆ รุ่นภาษาขนาดใหญ่" [2024-11] [กระดาษ]

เครื่องเข้ารหัส

1. PYMT5 (ช่วงการทุจริต): "PYMT5: การแปลหลายโหมดของภาษาธรรมชาติและรหัส Python ด้วยหม้อแปลง" [2020-10] [EMNLP 2020] [กระดาษ]

2. Mastropaolo และคณะ (MLM + deobfuscation): "DOBF: วัตถุประสงค์การฝึกอบรมล่วงหน้าสำหรับการเขียนโปรแกรมภาษา" [2021-02] [ICSE 2021] [กระดาษ] [repo]

3. DOBF (Span Corruption): "การศึกษาการใช้หม้อแปลงการถ่ายโอนข้อความเป็นข้อความเพื่อสนับสนุนงานที่เกี่ยวข้องกับรหัส" [2021-02] [Neurips 2021] [กระดาษ] [repo]

4. PLBART (DAE): "การฝึกอบรมล่วงหน้าเพื่อความเข้าใจและการสร้างโปรแกรม" [2021-03] [NAACL 2021] [กระดาษ] [repo]

5. codet5 (span corruption + identifier tagging + การทำนายตัวระบุหน้ากาก + text2Code + code2Text): "codet5: ตัวระบุตัวตนที่ได้รับการฝึกฝนก่อนที่จะได้รับการฝึกฝนล่วงหน้าสำหรับการทำความเข้าใจรหัสและการสร้าง" [2021-09]

6. SPT-Code (Span Corruption + NSP + การทำนายชื่อวิธี): "SPT-Code: ลำดับก่อนการฝึกอบรมสำหรับการเป็นตัวแทนซอร์สโค้ดการเรียนรู้" [2022-01] [ICSE 2022 ทางเทคนิคแทร็ก] [กระดาษ]

7. Alphacode (MLM + CLM): "การสร้างรหัสระดับการแข่งขันด้วย Alphacode" [2022-02] [วิทยาศาสตร์] [กระดาษ] [บล็อก]

8. Natgen (การแปลงสัญชาติโค้ด): "Natgen: การฝึกอบรมก่อนการกำเนิดโดย" การทำให้เป็นธรรมชาติ "ซอร์สโค้ด" [2022-06] [ESEC/FSE 2022] [กระดาษ] [repo]

9. Ernie-Code (Span Corruption + LM การแปลที่ใช้เดือย): "Ernie-Code: นอกเหนือจากภาษาอังกฤษที่เน้นภาษาอังกฤษเป็นศูนย์กลางสำหรับภาษาการเขียนโปรแกรม" [2022-12] [ACL23 (ผลการวิจัย)

10. Codet5 + (Span Corruption + CLM + การเรียนรู้แบบ code-code การเรียนรู้ + การแปลรหัสข้อความ): "codet5 +: เปิดรหัสแบบจำลองภาษาขนาดใหญ่สำหรับการทำความเข้าใจรหัสและการสร้าง" [2023-05] [EMNLP 2023] [กระดาษ] [repo]

11. AST-T5 (Span Corruption): "AST-T5: การเตรียมโครงสร้างให้ทราบถึงการสร้างรหัสและความเข้าใจ" [2024-01] [ICML 2024] [กระดาษ]

unilm

1. CUGLM (MLM + NSP + CLM): "รูปแบบภาษาที่ได้รับการฝึกอบรมมาหลายครั้งเพื่อการเรียนรู้แบบหลายงานสำหรับการกรอกรหัส" [2020-12] [ASE 2020] [กระดาษ]

2. UnixCoder (MLM + NSP + CLM + Span Corruption + การเรียนรู้แบบตรงกันข้าม + Code2Text): "UnixCoder: Unified Cross-Modal Pre-Training สำหรับการเป็นตัวแทนรหัส" [2022-03] [ACL 2022] [กระดาษ] [repo]

2.4 (คำแนะนำ) การปรับแต่งในรหัส

โมเดลเหล่านี้ใช้เทคนิคการปรับแต่งการเรียนการสอนเพื่อเพิ่มความสามารถของรหัส LLMS

1. WizardCoder (StarCoder + Evol-Instruct): "WizardCoder: เพิ่มขีดความสามารถรหัสภาษาขนาดใหญ่ที่มี Evol-Instruct" [2023-06] [ICLR 2024] [กระดาษ] [repo]

2. Pangu-Coder 2 (StarCoder + Evol-Instruct + RRTF): "Pangu-Coder2: การเพิ่มโมเดลภาษาขนาดใหญ่สำหรับรหัสที่มีข้อเสนอแนะการจัดอันดับ" [2023-07] [กระดาษ]

3. Octocoder (StarCoder) / Octogeex (codegeex2): "Octopack: รหัสคำสั่งการปรับแต่งโมเดลภาษาขนาดใหญ่" [2023-08] [ICLR 2024 Spotlight] [กระดาษ] [repo]

4. "ขั้นตอนการฝึกอบรมข้อมูลรหัสช่วย LLMS ให้เหตุผล" [2023-09] [ICLR 2024 Spotlight] [กระดาษ]

5. InstructCoder : "InstructCoder: คำแนะนำในการปรับแต่งแบบจำลองภาษาขนาดใหญ่สำหรับการแก้ไขรหัส" [กระดาษ] [repo]

6. MFTCoder : "MFTCODER: การเพิ่มรหัส LLMS ด้วยการปรับแต่งมัลติทาสก์" [2023-11] [KDD 2024] [กระดาษ] [repo]

7. "การทำความสะอาดรหัส LLM ช่วยสำหรับการฝึกอบรมเครื่องกำเนิดรหัสที่ถูกต้อง" [2023-11] [ICLR 2024] [กระดาษ]

8. Magicoder : "Magicoder: เสริมสร้างพลังการสร้างรหัสด้วย OSS-Instruct" [2023-12] [ICML 2024] [กระดาษ]

9. WaveCoder : "WaveCoder: การปรับปรุงที่แพร่หลายและหลากหลายสำหรับรหัสโมเดลภาษาขนาดใหญ่โดยการปรับแต่งคำสั่ง" [2023-12] [ACL 2024] [กระดาษ]

10. Astraios : "Astraios: พารามิเตอร์-ประสิทธิภาพการสอนการปรับแต่งรหัสภาษาขนาดใหญ่" [2024-01] [กระดาษ]

11. Dolphcoder : "Dolphcoder: รหัส echo-locating โมเดลภาษาขนาดใหญ่ที่มีการปรับแต่งการเรียนการสอนที่หลากหลายและหลากหลาย" [2024-02] [ACL 2024] [กระดาษ]

12. Safecoder : "คำสั่งการปรับแต่งสำหรับการสร้างรหัสที่ปลอดภัย" [2024-02] [ICML 2024] [กระดาษ]

13. "รหัสต้องการความคิดเห็น: การเพิ่มรหัส LLMS พร้อมการเพิ่มความคิดเห็น" [ACL 2024 การค้นพบ] [กระดาษ]

14. CCT : "การปรับการเปรียบเทียบรหัสสำหรับรหัสภาษาขนาดใหญ่" [2024-03] [กระดาษ]

15. SAT : "การปรับแต่งโครงสร้างอย่างละเอียดสำหรับโมเดลที่ผ่านการฝึกอบรมมาก่อนรหัส" [2024-04] [กระดาษ]

16. Codefort : "Codefort: การฝึกอบรมที่แข็งแกร่งสำหรับโมเดลการสร้างรหัส" [2024-04] [กระดาษ]

17. XFT : "XFT: ปลดล็อคพลังของการปรับแต่งรหัสโดยเพียงแค่รวมส่วนผสมของ experts" [2024-04] [ACL 2024] [กระดาษ] [repo]

18. AIEV-Instruct : "Autocoder: การเพิ่มรหัสโมเดลภาษาขนาดใหญ่ด้วย AIEV-Instruct" [2024-05] [กระดาษ]

19. AlchemistCoder : "AlchemistCoder: การประสานและความสามารถในการใช้รหัสโดยการปรับค่า Hindsight บนข้อมูลหลายแหล่ง" [2024-05] [กระดาษ]

20. "จากงานสัญลักษณ์ไปจนถึงการสร้างรหัส: การกระจายผลงานให้ผลงานที่ดีกว่า" [2024-05] [กระดาษ]

21. "การเปิดตัวผลกระทบของการเขียนโค้ดข้อมูลการปรับแต่งการปรับแต่งการให้เหตุผลแบบจำลองภาษาขนาดใหญ่" [2024-05] [กระดาษ]

22. พลัม : "พลัม: การเรียนรู้การตั้งค่าบวกกับกรณีทดสอบให้รูปแบบภาษารหัสที่ดีขึ้น" [2024-06] [กระดาษ]

23. McOder : "McEval: การประเมินรหัสหลายภาษาที่หลากหลาย" [2024-06] [กระดาษ]

24. "ปลดล็อกความสัมพันธ์ระหว่างการปรับแต่งการปรับแต่งและการเรียนรู้การเสริมแรงในรหัสการฝึกอบรมแบบจำลองภาษาขนาดใหญ่" [2024-06] [กระดาษ]

25. Code-Optimise : "รหัสที่เหมาะสม: ข้อมูลการตั้งค่าที่สร้างขึ้นด้วยตนเองเพื่อความถูกต้องและประสิทธิภาพ" [2024-06] [กระดาษ]

26. Unicoder : "Unicoder: การปรับรหัสแบบจำลองภาษาขนาดใหญ่ผ่านรหัสสากล" [2024-06] [ACL 2024] [กระดาษ]

27. "ความกะทัดรัดคือจิตวิญญาณของปัญญา: การตัดแต่งไฟล์ยาวสำหรับการสร้างรหัส" [2024-06] [กระดาษ]

28. "รหัสน้อยลงจัดตำแหน่งเพิ่มเติม: การปรับแต่ง LLM ที่มีประสิทธิภาพสำหรับการสร้างรหัสด้วยการตัดแต่งข้อมูล" [2024-07] [กระดาษ]

29. Inversecoder : "Inversecoder: ปลดปล่อยพลังของรหัสที่ปรับแต่งด้วยคำสั่ง LLMs ด้วย Instruct Instruct" [2024-07] [Paper]

30. "การเรียนรู้หลักสูตรสำหรับรูปแบบภาษารหัสขนาดเล็ก" [2024-07] [กระดาษ]

31. Instruct ทางพันธุกรรม : "คำสั่งทางพันธุกรรม: การปรับขนาดการสร้างคำแนะนำการเข้ารหัสสำหรับแบบจำลองภาษาขนาดใหญ่" [2024-07] [กระดาษ]

32. DataScope : "การสังเคราะห์ชุดข้อมูล Api-Guided ไปยังรูปแบบรหัสขนาดใหญ่" [2024-08] [กระดาษ]

33. ** xcoder **: "รหัส LLM ของคุณทำงานได้อย่างไร? เพิ่มขีดความสามารถในการปรับแต่งรหัสการปรับแต่งด้วยข้อมูลคุณภาพสูง" [2024-09] [กระดาษ]

34. Galla : "Galla: กราฟจัดเรียงโมเดลภาษาขนาดใหญ่สำหรับการทำความเข้าใจซอร์สโค้ดที่ดีขึ้น" [2024-09] [กระดาษ]

35. Hexacoder : "Hexacoder: การสร้างรหัสที่ปลอดภัยผ่านข้อมูลการฝึกอบรมสังเคราะห์แบบออราเคิล-ไกด์" [2024-09] [กระดาษ]

36. AMR-EVOL : "AMR-EVOL: การตอบสนองแบบแยกส่วนวิวัฒนาการ Evolution ทำให้เกิดการกลั่นความรู้ที่ดีขึ้นสำหรับแบบจำลองภาษาขนาดใหญ่ในการสร้างรหัส" [2024-10] [กระดาษ]

37. LINTSEQ : "แบบจำลองภาษาการฝึกอบรมเกี่ยวกับลำดับการแก้ไขสังเคราะห์ปรับปรุงการสังเคราะห์รหัส" [2024-10] [กระดาษ]

38. Coba : "Coba: Convergence Balancer สำหรับ Multitask Finetuning ของรุ่นภาษาขนาดใหญ่" [2024-10] [EMNLP 2024] [กระดาษ]

39. Cursorcore : "Cursorcore: ช่วยเหลือการเขียนโปรแกรมผ่านการจัดตำแหน่งอะไรก็ได้" [2024-10] [กระดาษ]

40. SelfCodealign : "SelfCodealign: การจัดตำแหน่งตนเองสำหรับการสร้างรหัส" [2024-10] [กระดาษ]

41. "การควบคุมงานฝีมือของการสังเคราะห์ข้อมูลสำหรับ codellms" [2024-10] [กระดาษ]

42. Codelutra : "Codelutra: การเพิ่มการสร้างรหัส LLM ผ่านการปรับแต่งการปรับแต่ง" [2024-11] [กระดาษ]

43. DSTC : "DSTC: การเรียนรู้การตั้งค่าโดยตรงด้วยการทดสอบและรหัสที่สร้างขึ้นเองเท่านั้นเพื่อปรับปรุงรหัส LMS" [2024-11] [กระดาษ]

2.5 การเรียนรู้การเสริมแรงทางรหัส

1. compcoder : "การสร้างรหัสประสาทที่รวบรวมได้ด้วยคำติชมของคอมไพเลอร์" [2022-03] [ACL 2022] [กระดาษ]

2. Coderl : "Coderl: การสร้างรหัสการเรียนรู้ผ่านแบบจำลองที่ผ่านการฝึกอบรมและการเรียนรู้การเสริมแรงอย่างลึกซึ้ง" [2022-07] [Neurips 2022] [Paper] [repo]

3. PPOCODER : "การสร้างรหัสตามการดำเนินการโดยใช้การเรียนรู้การเสริมแรงลึก" [2023-01] [TMLR 2023] [กระดาษ] [repo]

4. RLTF : "RLTF: การเรียนรู้การเสริมแรงจากข้อเสนอแนะการทดสอบหน่วย" [2023-07] [กระดาษ] [repo]

5. B-coder : "B-coder: การเรียนรู้การเสริมแรงลึกตามมูลค่าสำหรับการสังเคราะห์โปรแกรม" [2023-10] [ICLR 2024] [กระดาษ]

6. Ircoco : "Ircoco: การเรียนรู้การเสริมแรงอย่างลึกซึ้งในการเรียนรู้การเสริมกำลังเพื่อการจบรหัส" [2024-01] [FSE 2024] [กระดาษ]

7. stepcoder : "stepcoder: ปรับปรุงการสร้างรหัสด้วยการเรียนรู้การเสริมแรงจากคำติชมคอมไพเลอร์" [2024-02] [ACL 2024] [กระดาษ]

8. RLPF & DPA : "LLM ที่สอดคล้องกับประสิทธิภาพสำหรับการสร้างรหัสที่รวดเร็ว" [2024-04] [กระดาษ]

9. "การวัดการท่องจำใน RLHF สำหรับการกรอกรหัส" [2024-06] [กระดาษ]

10. "การใช้ RLAIF สำหรับการสร้างรหัสด้วย API-USAGE ใน LLM ที่มีน้ำหนักเบา" [2024-06] [กระดาษ]

11. RLCODER : "RLCODER: การเรียนรู้การเสริมแรงสำหรับการกรอกรหัสระดับที่เก็บ" [2024-07] [กระดาษ]

12. PF-PPO : "การกรองนโยบายใน RLHF เพื่อปรับแต่ง LLM สำหรับการสร้างรหัส" [2024-09] [กระดาษ]

13. Coffee-Gym : "Coffee-Gym: สภาพแวดล้อมสำหรับการประเมินและปรับปรุงข้อเสนอแนะภาษาธรรมชาติเกี่ยวกับรหัสที่ผิดพลาด" [2024-09] [กระดาษ]

14. RLEF : "RLEF: การลงดินรหัส LLMS ในการดำเนินการตอบรับด้วยการเรียนรู้การเสริมแรง" [2024-10] [กระดาษ]

15. codepmp : "codepmp: รูปแบบการตั้งค่าที่ปรับขนาดได้ก่อนการเตรียมการสำหรับการใช้เหตุผลแบบจำลองภาษาขนาดใหญ่" [2024-10] [กระดาษ]

16. CodedPo : "CodedPo: การจัดแนวโมเดลการจัดแนวด้วยซอร์สโค้ดที่สร้างขึ้นและตรวจสอบด้วยตนเอง" [2024-10] [กระดาษ]

17. "การเพิ่มประสิทธิภาพนโยบายการกำกับดูแลการกำกับดูแลสำหรับการสร้างรหัส" [2024-10] [กระดาษ]

18. "จัดตำแหน่ง codellms กับการเพิ่มประสิทธิภาพการตั้งค่าโดยตรง" [2024-10] [กระดาษ]

19. Falcon : "Falcon: ระบบปรับแต่งการเพิ่มประสิทธิภาพการเข้ารหัสแบบยาว/ระยะสั้นที่ขับเคลื่อนด้วยการตอบกลับ" [2024-10] [กระดาษ]

20. PFPO : "การเพิ่มประสิทธิภาพการตั้งค่าสำหรับการให้เหตุผลกับคำติชมแบบหลอก" [2024-11] [กระดาษ]

3. เมื่อการเขียนโค้ดตรงตามการใช้เหตุผล

3.1 การเข้ารหัสเพื่อการให้เหตุผล

1. PAL : "PAL: โมเดลภาษาโดยโปรแกรมช่วย" [2022-11] [ICML 2023] [กระดาษ] [repo]

2. POT : "โปรแกรมการกระตุ้นความคิด: การคำนวณการคำนวณจากการใช้เหตุผลสำหรับงานการใช้เหตุผลเชิงตัวเลข" [2022-11] [TMLR 2023] [กระดาษ] [repo]

3. PAD : "PAD: การกลั่นด้วยโปรแกรมช่วยสอนรุ่นเล็ก ๆ ให้เหตุผลได้ดีกว่าการปรับแต่งโซ่ของการปรับแต่ง" [2023-05] [NAACL 2024] [กระดาษ]

4. CSV : "การแก้ปัญหาคำศัพท์ทางคณิตศาสตร์ที่ท้าทายโดยใช้ล่ามโค้ด GPT-4 ด้วยการตรวจสอบด้วยตนเองตามรหัส" [2023-08] [ICLR 2024] [กระดาษ]

5. MathCoder : "MathCoder: การรวมรหัสไร้รอยต่อใน LLMS เพื่อเพิ่มการใช้เหตุผลทางคณิตศาสตร์" [2023-10] [ICLR 2024] [กระดาษ]

6. COC : "Chain of Code: การใช้เหตุผลกับ Emulator รหัสแบบจำลองภาษา" [2023-12] [ICML 2024] [กระดาษ]

7. มาริโอ : "มาริโอ: การใช้เหตุผลทางคณิตศาสตร์ด้วยเอาท์พุทโค้ดล่าม-ไปป์ไลน์ที่ทำซ้ำได้" [2024-01] [ACL 2024 การค้นพบ] [กระดาษ]

8. Regal : "Regal: โปรแกรม refactoring เพื่อค้นหา abstractions ทั่วไป" [2024-01] [ICML 2024] [กระดาษ]

9. "การดำเนินการรหัสปฏิบัติการทำให้เกิดตัวแทน LLM ที่ดีกว่า" [2024-02] [ICML 2024] [กระดาษ]

10. HPROPRO : "การสำรวจคำถามไฮบริดผ่านการแจ้งเตือนด้วยโปรแกรม" [2024-02] [ACL 2024] [กระดาษ]

11. Xstreet : "ทำให้การใช้เหตุผลเชิงโครงสร้างหลายภาษาดีขึ้นจาก LLMs ผ่านรหัส" [2024-03] [ACL 2024] [กระดาษ]

12. FlowMind : "FlowMind: การสร้างเวิร์กโฟลว์อัตโนมัติด้วย LLMS" [2024-03] [กระดาษ]

13. Think-and-Execute : "แบบจำลองภาษาเป็นคอมไพเลอร์: การจำลองการดำเนินการ pseudocode ช่วยปรับปรุงการใช้เหตุผลอัลกอริทึมในรูปแบบภาษา" [2024-04] [กระดาษ]

14. Core : "Core: LLM เป็นล่ามสำหรับการเขียนโปรแกรมภาษาธรรมชาติการเขียนโปรแกรมรหัสหลอกและการเขียนโปรแกรมการไหลของตัวแทน AI" [2024-05] [กระดาษ]

15. Mumath-Code : "Mumath-Code: การรวมโมเดลภาษาขนาดใหญ่ใช้เครื่องมือกับการเพิ่มข้อมูลหลายมุมมองสำหรับการใช้เหตุผลทางคณิตศาสตร์" [2024-05] [กระดาษ]

16. Cogex : "การเรียนรู้ที่จะให้เหตุผลผ่านการสร้างโปรแกรมการจำลองและการค้นหา" [2024-05] [กระดาษ]

17. "การใช้เหตุผลทางคณิตศาสตร์ด้วย LLM: การสร้างและการเปลี่ยนแปลง" [2024-05] [กระดาษ]

18. "LLMs สามารถให้เหตุผลในป่าด้วยโปรแกรมได้หรือไม่" [2024-06] [กระดาษ]

19. Dotamath : "Dotamath: การสลายตัวของความคิดด้วยความช่วยเหลือรหัสและการแก้ไขตนเองสำหรับการใช้เหตุผลทางคณิตศาสตร์" [2024-07] [กระดาษ]

20. Cibench : "Cibench: การประเมิน LLM ของคุณด้วยปลั๊กอินโค้ดล่าม" [2024-07] [กระดาษ]

21. Pybench : "Pybench: การประเมิน Agent LLM ในงานการเข้ารหัสในโลกแห่งความเป็นจริงต่างๆ" [2024-07] [กระดาษ]

22. Adacoder : "Adacoder: Adaptive Promption การบีบอัดสำหรับการตอบคำถามด้วยภาพแบบโปรแกรม" [2024-07] [กระดาษ]

23. PyramidCoder : "Pyramid Coder: ตัวสร้างรหัสลำดับชั้นสำหรับการตอบคำถามด้วยภาพประกอบ" [2024-07] [กระดาษ]

24. CodeGraph : "CodeGraph: การเพิ่มเหตุผลกราฟของ LLMS ด้วยรหัส" [2024-08] [กระดาษ]

25. SIAM : "SIAM: การปรับปรุงตัวเองด้วยการให้เหตุผลทางคณิตศาสตร์ด้วยความช่วยเหลือทางคณิตศาสตร์ของแบบจำลองภาษาขนาดใหญ่" [2024-08] [กระดาษ]

26. CodePlan : "CodePlan: ปลดล็อคการใช้เหตุผลที่มีศักยภาพในรุ่น Langauge ขนาดใหญ่โดยการปรับขนาดการวางแผนรูปแบบรหัส" [2024-09] [กระดาษ]

27. POT : "การพิสูจน์ความคิด: การสังเคราะห์โปรแกรม neurosymbolic ช่วยให้การใช้เหตุผลที่แข็งแกร่งและตีความได้" [2024-09] [กระดาษ]

28. Metamath : "Metamath: การรวมภาษาธรรมชาติและรหัสเพื่อการใช้เหตุผลทางคณิตศาสตร์ที่เพิ่มขึ้นในแบบจำลองภาษาขนาดใหญ่" [2024-09] [กระดาษ]

29. "Babelbench: มาตรฐาน OMNI สำหรับการวิเคราะห์รหัสที่ขับเคลื่อนด้วยรหัสของข้อมูลหลายรูปแบบและหลายโครงสร้าง" [2024-10] [กระดาษ]

30. Codesteer : "การควบคุมแบบจำลองภาษาขนาดใหญ่ระหว่างการดำเนินการรหัสและการใช้เหตุผลเชิงข้อความ" [2024-10] [กระดาษ]

31. MathCoder2 : "MathCoder2: การใช้เหตุผลทางคณิตศาสตร์ที่ดีขึ้นจากการเตรียมการอย่างต่อเนื่องเกี่ยวกับรหัสคณิตศาสตร์แบบแปลนแบบจำลอง" [2024-10] [กระดาษ]

32. LLMFP : "การวางแผนอะไรก็ตามที่มีความเข้มงวด: การวางแผนแบบศูนย์อเนกประสงค์ด้วยการเขียนโปรแกรมอย่างเป็นทางการที่ใช้ LLM" [2024-10] [กระดาษ]

33. พิสูจน์ว่า : "ไม่ได้รับคะแนนเสียงทั้งหมด!

34. พิสูจน์ว่า : "ความน่าเชื่อถือ แต่ตรวจสอบ: การประเมิน VLM แบบโปรแกรมในป่า" [2024-10] [กระดาษ]

35. Geocoder : "Geocoder: การแก้ปัญหาเรขาคณิตโดยการสร้างรหัสโมดูลาร์ผ่านแบบจำลอง Vision-Language" [2024-10] [กระดาษ]

36. เหตุผล : "เหตุผล: การใช้โปรแกรมสัญลักษณ์ที่สกัดได้เพื่อประเมินการใช้เหตุผลทางคณิตศาสตร์" [2024-10] [กระดาษ]

37. GFP : "การกระตุ้นการเติมช่องว่างช่วยเพิ่มการใช้เหตุผลทางคณิตศาสตร์ด้วยรหัส" [2024-11] [กระดาษ]

38. Utmath : "Utmath: การประเมินคณิตศาสตร์ด้วยการทดสอบหน่วยผ่านความคิดที่ให้เหตุผลถึงการเข้ารหัส" [2024-11] [กระดาษ]

39. COCOP : "COCOP: การเพิ่มการจำแนกประเภทข้อความด้วย LLM ผ่าน Promption Code Prompt" [2024-11] [กระดาษ]

40. REPL-PLAN : "การวางแผนการเสริมรหัสแบบโต้ตอบและการแสดงออกด้วยรูปแบบภาษาขนาดใหญ่" [2024-11] [กระดาษ]

3.2 การจำลองรหัส

• "ความท้าทายในการจำลองรหัสสำหรับแบบจำลองภาษาขนาดใหญ่" [2024-01] [กระดาษ]

• "CodeMind: กรอบการทำงานที่ท้าทายรูปแบบภาษาขนาดใหญ่สำหรับการให้เหตุผลรหัส" [2024-02] [กระดาษ]

• "การดำเนินการอัลกอริทึมที่อธิบายภาษาธรรมชาติด้วยแบบจำลองภาษาขนาดใหญ่: การสอบสวน" [2024-02] [กระดาษ]

• "โมเดลภาษาสามารถแกล้งทำเป็นแก้ปัญหารหัสลอจิกด้วย LLMS" [2024-03] [กระดาษ]

• "การประเมินแบบจำลองภาษาขนาดใหญ่ที่มีพฤติกรรมรันไทม์ของการดำเนินการโปรแกรม" [2024-03] [กระดาษ]

• "ถัดไป: การสอนแบบจำลองภาษาขนาดใหญ่เพื่อเหตุผลเกี่ยวกับการดำเนินการรหัส" [2024-04] [ICML 2024] [กระดาษ]

• "Selfpico: การดำเนินการรหัสบางส่วนด้วยตนเองด้วย LLMS" [2024-07] [กระดาษ]

• "แบบจำลองภาษาขนาดใหญ่เป็นผู้ดำเนินการรหัส: การศึกษาเชิงสำรวจ" [2024-10] [กระดาษ]

• "VisualCoder: ชี้นำโมเดลภาษาขนาดใหญ่ในการดำเนินการรหัสด้วยการใช้เหตุผลหลายรูปแบบที่มีความคิดหลายรูปแบบ" [2024-10] [กระดาษ]

3.3 CODE AGENTS

1. การรวบรวมตนเอง : "การสร้างรหัสการรวมตัวเองผ่าน chatgpt" [2023-04] [กระดาษ]

2. chatdev : "ตัวแทนสื่อสารเพื่อการพัฒนาซอฟต์แวร์" [2023-07] [กระดาษ] [repo]

3. MetAgpt : "MetAgpt: การเขียนโปรแกรมเมตาสำหรับกรอบการทำงานร่วมกันหลายตัวแทน" [2023-08] [กระดาษ] [repo]

4. Codechain : "Codechain: ไปสู่การสร้างรหัสแบบแยกส่วนผ่านห่วงโซ่การแก้ไขตนเองด้วยโมดูลย่อยตัวแทน" [2023-10] [ICLR 2024] [กระดาษ]

5. CodeAgent : "CodeAgent: การเพิ่มการสร้างรหัสด้วยระบบตัวแทนแบบรวมเครื่องมือสำหรับความท้าทายในการเข้ารหัสระดับ Repo ระดับโลก" [2024-01] [ACL 2024] [กระดาษ]

6. Conline : "Conline: การสร้างรหัสที่ซับซ้อนและการปรับแต่งด้วยการทดสอบการค้นหาออนไลน์และการทดสอบความถูกต้อง" [2024-03] [กระดาษ]

7. LCG : "เมื่อการสร้างรหัสที่ใช้ LLM เป็นไปตามกระบวนการพัฒนาซอฟต์แวร์" [2024-03] [กระดาษ]

8. repairagent : "repairagent: ตัวแทนอิสระ, LLM สำหรับการซ่อมแซมโปรแกรม" [2024-03] [กระดาษ]

9. Magis :: "Magis: กรอบหลายตัวแทนที่ใช้ LLM สำหรับการแก้ไขปัญหาปัญหา GitHub" [2024-03] [กระดาษ]

10. SOA : "ตัวแทนที่จัดระเบียบตนเอง: LLM Multi-Agent Framework ไปสู่การสร้างรหัสขนาดใหญ่พิเศษและการเพิ่มประสิทธิภาพ" [2024-04] [กระดาษ]

11. Autocoderover : "Autocoderover: การปรับปรุงโปรแกรมอัตโนมัติ" [2024-04] [กระดาษ]

12. SWE-Agent : "SWE-Agent: อินเตอร์เฟสตัวแทนคอมพิวเตอร์เปิดใช้งานวิศวกรรมซอฟต์แวร์อัตโนมัติ" [2024-05] [กระดาษ]

13. MAPCODER : "MAPCODER: การสร้างรหัสหลายตัวแทนสำหรับการแก้ปัญหาการแข่งขัน" [2024-05] [ACL 2024] [กระดาษ]

14. "Fight Fire With Fire: เราจะไว้วางใจ CHATGPT ได้มากแค่ไหนในงานที่เกี่ยวข้องกับรหัสแหล่งที่มา" [2024-05] [กระดาษ]

15. FunCoder : "Divide-and-Conquer Meets Consensus: Unleashing the Power of Functions in Code Generation" [2024-05] [paper]

16. CTC : "Multi-Agent Software Development through Cross-Team Collaboration" [2024-06] [paper]

17. MASAI : "MASAI: Modular Architecture for Software-engineering AI Agents" [2024-06] [paper]

18. AgileCoder : "AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology" [2024-06] [paper]

19. CodeNav : "CodeNav: Beyond tool-use to using real-world codebases with LLM agents" [2024-06] [paper]

20. INDICT : "INDICT: Code Generation with Internal Dialogues of Critiques for Both Security and Helpfulness" [2024-06] [paper]

21. AppWorld : "AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents" [2024-07] [paper]

22. CortexCompile : "CortexCompile: Harnessing Cortical-Inspired Architectures for Enhanced Multi-Agent NLP Code Synthesis" [2024-08] [paper]

23. Survey : "Large Language Model-Based Agents for Software Engineering: A Survey" [2024-09] [paper]

24. AutoSafeCoder : "AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing" [2024-09] [paper]

25. SuperCoder2.0 : "SuperCoder2.0: Technical Report on Exploring the feasibility of LLMs as Autonomous Programmer" [2024-09] [paper]

26. Survey : "Agents in Software Engineering: Survey, Landscape, and Vision" [2024-09] [paper]

27. MOSS : "MOSS: Enabling Code-Driven Evolution and Context Management for AI Agents" [2024-09] [paper]

28. HyperAgent : "HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale" [2024-09] [paper]

29. "Compositional Hardness of Code in Large Language Models -- A Probabilistic Perspective" [2024-09] [paper]

30. RGD : "RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance" [2024-10] [paper]

31. AutoML-Agent : "AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML" [2024-10] [paper]

32. Seeker : "Seeker: Enhancing Exception Handling in Code with LLM-based Multi-Agent Approach" [2024-10] [paper]

33. REDO : "REDO: Execution-Free Runtime Error Detection for COding Agents" [2024-10] [paper]

34. "Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios" [2024-10] [paper]

35. EvoMAC : "Self-Evolving Multi-Agent Collaboration Networks for Software Development" [2024-10] [paper]

36. VisionCoder : "VisionCoder: Empowering Multi-Agent Auto-Programming for Image Processing with Hybrid LLMs" [2024-10] [paper]

37. AutoKaggle : "AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions" [2024-10] [paper]

38. Watson : "Watson: A Cognitive Observability Framework for the Reasoning of Foundation Model-Powered Agents" [2024-11] [paper]

39. CodeTree : "CodeTree: Agent-guided Tree Search for Code Generation with Large Language Models" [2024-11] [paper]

40. EvoCoder : "LLMs as Continuous Learners: Improving the Reproduction of Defective Code in Software Issues" [2024-11] [paper]

3.4 Interactive Coding

• "Interactive Program Synthesis" [2017-03] [paper]

• "Question selection for interactive program synthesis" [2020-06] [PLDI 2020] [paper]

• "Interactive Code Generation via Test-Driven User-Intent Formalization" [2022-08] [paper]

• "Improving Code Generation by Training with Natural Language Feedback" [2023-03] [TMLR] [paper]

• "Self-Refine: Iterative Refinement with Self-Feedback" [2023-03] [NeurIPS 2023] [paper]

• "Teaching Large Language Models to Self-Debug" [2023-04] [paper]

• "Self-Edit: Fault-Aware Code Editor for Code Generation" [2023-05] [ACL 2023] [paper]

• "LeTI: Learning to Generate from Textual Interactions" [2023-05] [paper]

• "Is Self-Repair a Silver Bullet for Code Generation?" [2023-06] [ICLR 2024] [paper]

• "InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback" [2023-06] [NeurIPS 2023] [paper]

• "INTERVENOR: Prompting the Coding Ability of Large Language Models with the Interactive Chain of Repair" [2023-11] [ACL 2024 Findings] [paper]

• "OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement" [2024-02] [ACL 2024 Findings] [paper]

• "Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback" [2024-03] [ACL 2024 Findings] [paper]

• "CYCLE: Learning to Self-Refine the Code Generation" [2024-03] [paper]

• "LLM-based Test-driven Interactive Code Generation: User Study and Empirical Evaluation" [2024-04] [paper]

• "SOAP: Enhancing Efficiency of Generated Code via Self-Optimization" [2024-05] [paper]

• "Code Repair with LLMs gives an Exploration-Exploitation Tradeoff" [2024-05] [paper]

• "ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generation" [2024-05] [paper]

• "Training LLMs to Better Self-Debug and Explain Code" [2024-05] [paper]

• "Requirements are All You Need: From Requirements to Code with LLMs" [2024-06] [paper]

• "I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation" [2024-07] [paper]

• "An Empirical Study on Self-correcting Large Language Models for Data Science Code Generation" [2024-08] [paper]

• "RethinkMCTS: Refining Erroneous Thoughts in Monte Carlo Tree Search for Code Generation" [2024-09] [paper]

• "From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging" [2024-10] [paper] [repo]

• "What Makes Large Language Models Reason in (Multi-Turn) Code Generation?" [2024-10] [paper]

• "The First Prompt Counts the Most! An Evaluation of Large Language Models on Iterative Example-based Code Generation" [2024-11] [paper]

• "Planning-Driven Programming: A Large Language Model Programming Workflow" [2024-11] [paper]

• "ConAIR:Consistency-Augmented Iterative Interaction Framework to Enhance the Reliability of Code Generation" [2024-11] [paper]

3.5 Frontend Navigation

• "MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding" [2021-10] [ACL 2022] [paper]

• "WebKE: Knowledge Extraction from Semi-structured Web with Pre-trained Markup Language Model" [2021-10] [CIKM 2021] [paper]

• "WebGPT: Browser-assisted question-answering with human feedback" [2021-12] [paper]

• "CM3: A Causal Masked Multimodal Model of the Internet" [2022-01] [paper]

• "DOM-LM: Learning Generalizable Representations for HTML Documents" [2022-01] [paper]

• "WebFormer: The Web-page Transformer for Structure Information Extraction" [2022-02] [WWW 2022] [paper]

• "A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility" [2022-02] [ECCV 2022] [paper]

• "WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents" [2022-07] [NeurIPS 2022] [paper]

• "Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding" [2022-10] [ICML 2023] [paper]

• "Understanding HTML with Large Language Models" [2022-10] [EMNLP 2023 findings] [paper]

• "WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics" [2023-01] [CHI 2023] [paper]

• "Mind2Web: Towards a Generalist Agent for the Web" [2023-06] [NeurIPS 2023] [paper]

• "A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis", [2023-07] [ICLR 2024] [paper]

• "WebArena: A Realistic Web Environment for Building Autonomous Agents" [2023-07] [paper]

• "CogAgent: A Visual Language Model for GUI Agents" [2023-12] [paper]

• "GPT-4V(ision) is a Generalist Web Agent, if Grounded" [2024-01] [paper]

• "WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models" [2024-01] [paper]

• "WebLINX: Real-World Website Navigation with Multi-Turn Dialogue" [2024-02] [paper]

• "OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web" [2024-02] [paper]

• "AutoWebGLM: Bootstrap And Reinforce A Large Language Model-based Web Navigating Agent" [2024-04] [paper]

• "WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents" [2024-04] [paper]

• "AutoCrawler: A Progressive Understanding Web Agent for Web Crawler Generation" [2024-04] [paper]

• "GUICourse: From General Vision Language Models to Versatile GUI Agents" [2024-06] [paper]

• "NaviQAte: Functionality-Guided Web Application Navigation" [2024-09] [paper]

• "MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding" [2024-09] [paper]

• "Multimodal Auto Validation For Self-Refinement in Web Agents" [2024-10] [paper]

• "Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents" [2024-10] [paper]

• "Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation" [2024-10] [paper]

• "Harnessing Webpage UIs for Text-Rich Visual Understanding" [2024-10] [paper]

• "AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents" [2024-10] [paper]

• "Beyond Browsing: API-Based Web Agents" [2024-10] [paper]

• "Large Language Models Empowered Personalized Web Agents" [2024-10] [paper]

• "AdvWeb: Controllable Black-box Attacks on VLM-powered Web Agents" [2024-10] [paper]

• "Auto-Intent: Automated Intent Discovery and Self-Exploration for Large Language Model Web Agents" [2024-10] [paper]

• "OS-ATLAS: A Foundation Action Model for Generalist GUI Agents" [2024-10] [paper]

• "From Context to Action: Analysis of the Impact of State Representation and Context on the Generalization of Multi-Turn Web Navigation Agents" [2024-10] [paper]

• "AutoGLM: Autonomous Foundation Agents for GUIs" [2024-10] [paper]

• "WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning" [2024-11] [paper]

• "The Dawn of GUI Agent: A Preliminary Case Study with Claude 3.5 Computer Use" [2024-11] [paper]

• "ScribeAgent: Towards Specialized Web Agents Using Production-Scale Workflow Data" [2024-11] [paper]

• "ShowUI: One Vision-Language-Action Model for GUI Visual Agent" [2024-11] [paper]

4. Code LLM for Low-Resource, Low-Level, and Domain-Specific Languages

• [ Ruby ] "On the Transferability of Pre-trained Language Models for Low-Resource Programming Languages" [2022-04] [ICPC 2022] [paper]

• [ Verilog ] "Benchmarking Large Language Models for Automated Verilog RTL Code Generation" [2022-12] [DATE 2023] [paper]

• [ OCL ] "On Codex Prompt Engineering for OCL Generation: An Empirical Study" [2023-03] [MSR 2023] [paper]

• [ Ansible-YAML ] "Automated Code generation for Information Technology Tasks in YAML through Large Language Models" [2023-05] [DAC 2023] [paper]

• [ Hansl ] "The potential of LLMs for coding with low-resource and domain-specific programming languages" [2023-07] [paper]

• [ Verilog ] "VeriGen: A Large Language Model for Verilog Code Generation" [2023-07] [paper]

• [ Verilog ] "RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model" [2023-08] [paper]

• [ Racket, OCaml, Lua, R, Julia ] "Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs" [2023-08] [paper]

• [ Verilog ] "VerilogEval: Evaluating Large Language Models for Verilog Code Generation" [2023-09] [ICCAD 2023] [paper]

• [ Verilog ] "RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Models" [2023-11] [paper]

• [ Verilog ] "Advanced Large Language Model (LLM)-Driven Verilog Development: Enhancing Power, Performance, and Area Optimization in Code Synthesis" [2023-12] [paper]

• [ Verilog ] "RTLCoder: Outperforming GPT-3.5 in Design RTL Generation with Our Open-Source Dataset and Lightweight Solution" [2023-12] [paper]

• [ Verilog ] "BetterV: Controlled Verilog Generation with Discriminative Guidance" [2024-02] [ICML 2024] [paper]

• [ R ] "Empirical Studies of Parameter Efficient Methods for Large Language Models of Code and Knowledge Transfer to R" [2024-03] [paper]

• [ Haskell ] "Investigating the Performance of Language Models for Completing Code in Functional Programming Languages: a Haskell Case Study" [2024-03] [paper]

• [ Verilog ] "A Multi-Expert Large Language Model Architecture for Verilog Code Generation" [2024-04] [paper]

• [ Verilog ] "CreativEval: Evaluating Creativity of LLM-Based Hardware Code Generation" [2024-04] [paper]

• [ Alloy ] "An Empirical Evaluation of Pre-trained Large Language Models for Repairing Declarative Formal Specifications" [2024-04] [paper]

• [ Verilog ] "Evaluating LLMs for Hardware Design and Test" [2024-04] [paper]

• [ Kotlin, Swift, and Rust ] "Software Vulnerability Prediction in Low-Resource Languages: An Empirical Study of CodeBERT and ChatGPT" [2024-04] [paper]

• [ Verilog ] "MEIC: Re-thinking RTL Debug Automation using LLMs" [2024-05] [paper]

• [ Bash ] "Tackling Execution-Based Evaluation for NL2Bash" [2024-05] [paper]

• [ Fortran, Julia, Matlab, R, Rust ] "Evaluating AI-generated code for C++, Fortran, Go, Java, Julia, Matlab, Python, R, and Rust" [2024-05] [paper]

• [ OpenAPI ] "Optimizing Large Language Models for OpenAPI Code Completion" [2024-05] [paper]

• [ Kotlin ] "Kotlin ML Pack: Technical Report" [2024-05] [paper]

• [ Verilog ] "VerilogReader: LLM-Aided Hardware Test Generation" [2024-06] [paper]

• "Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming" [2024-06] [paper]

• [ Logo ] "Program Synthesis Benchmark for Visual Programming in XLogoOnline Environment" [2024-06] [paper]

• [ Ansible YAML, Bash ] "DocCGen: Document-based Controlled Code Generation" [2024-06] [paper]

• [ Qiskit ] "Qiskit HumanEval: An Evaluation Benchmark For Quantum Code Generative Models" [2024-06] [paper]

• [ Perl, Golang, Swift ] "DistiLRR: Transferring Code Repair for Low-Resource Programming Languages" [2024-06] [paper]

• [ Verilog ] "AssertionBench: A Benchmark to Evaluate Large-Language Models for Assertion Generation" [2024-06] [paper]

• "A Comparative Study of DSL Code Generation: Fine-Tuning vs. Optimized Retrieval Augmentation" [2024-07] [paper]

• [ Json, XLM, YAML ] "ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages" [2024-07] [paper]

• [ Verilog ] "AutoBench: Automatic Testbench Generation and Evaluation Using LLMs for HDL Design" [2024-07] [paper]

• [ Verilog ] "CodeV: Empowering LLMs for Verilog Generation through Multi-Level Summarization" [2024-07] [paper]

• [ Verilog ] "ITERTL: An Iterative Framework for Fine-tuning LLMs for RTL Code Generation" [2024-07] [paper]

• [ Verilog ] "OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflection" [2024-07] [paper]

• [ Verilog ] "Large Language Model for Verilog Generation with Golden Code Feedback" [2024-07] [paper]

• [ Verilog ] "AutoVCoder: A Systematic Framework for Automated Verilog Code Generation using LLMs" [2024-07] [paper]

• [ RPA ] "Plan with Code: Comparing approaches for robust NL to DSL generation" [2024-08] [paper]

• [ Verilog ] "VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool" [2024-08] [paper]

• [ Verilog ] "Revisiting VerilogEval: Newer LLMs, In-Context Learning, and Specification-to-RTL Tasks" [2024-08] [paper]

• [ MaxMSP, Web Audio ] "Benchmarking LLM Code Generation for Audio Programming with Visual Dataflow Languages" [2024-09] [paper]

• [ Verilog ] "RTLRewriter: Methodologies for Large Models aided RTL Code Optimization" [2024-09] [paper]

• [ Verilog ] "CraftRTL: High-quality Synthetic Data Generation for Verilog Code Models with Correct-by-Construction Non-Textual Representations and Targeted Code Repair" [2024-09] [paper]

• [ Bash ] "ScriptSmith: A Unified LLM Framework for Enhancing IT Operations via Automated Bash Script Generation, Assessment, and Refinement" [2024-09] [paper]

• [ Survey ] "Survey on Code Generation for Low resource and Domain Specific Programming Languages" [2024-10] [paper]

• [ R ] "Do Current Language Models Support Code Intelligence for R Programming Language?" [2024-10] [paper]

• "Can Large Language Models Generate Geospatial Code?" [2024-10] [paper]

• [ PLC ] "Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agents" [2024-10] [paper]

• [ Lua ] "Evaluating Quantized Large Language Models for Code Generation on Low-Resource Language Benchmarks" [2024-10] [paper]

• "Improving Parallel Program Performance Through DSL-Driven Code Generation with LLM Optimizers" [2024-10] [paper]

• "GeoCode-GPT: A Large Language Model for Geospatial Code Generation Tasks" [2024-10] [paper]

• [ R, D, Racket, Bash ]: "Bridge-Coder: Unlocking LLMs' Potential to Overcome Language Gaps in Low-Resource Code" [2024-10] [paper]

• [ SPICE ]: "SPICEPilot: Navigating SPICE Code Generation and Simulation with AI Guidance" [2024-10] [paper]

• [ IEC 61131-3 ST ]: "Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback" [2024-10] [paper]

• [ Verilog ] "MetRex: A Benchmark for Verilog Code Metric Reasoning Using LLMs" [2024-11] [paper]

• [ Verilog ] "CorrectBench: Automatic Testbench Generation with Functional Self-Correction using LLMs for HDL Design" [2024-11] [paper]

• [ MUMPS, ALC ] "Leveraging LLMs for Legacy Code Modernization: Challenges and Opportunities for LLM-Generated Documentation" [2024-11] [paper]

5. Methods/Models for Downstream Tasks

For each task, the first column contains non-neural methods (eg n-gram, TF-IDF, and (occasionally) static program analysis); the second column contains non-Transformer neural methods (eg LSTM, CNN, GNN); the third column contains Transformer based methods (eg BERT, GPT, T5).

Code Generation

• "Enhancing Large Language Models in Coding Through Multi-Perspective Self-Consistency" [2023-09] [ACL 2024] [paper]

• "Self-Infilling Code Generation" [2023-11] [ICML 2024] [paper]

• "JumpCoder: Go Beyond Autoregressive Coder via Online Modification" [2024-01] [ACL 2024] [paper]

• "Unsupervised Evaluation of Code LLMs with Round-Trip Correctness" [2024-02] [ICML 2024] [paper]

• "The Larger the Better? Improved LLM Code-Generation via Budget Reallocation" [2024-03] [paper]

• "Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models" [2024-03] [ACL 2024] [paper]

• "Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective" [2024-04] [ACL 2024 Findings] [paper]

• "Distilling Algorithmic Reasoning from LLMs via Explaining Solution Programs" [2024-04] [paper]

• "Quality Assessment of Prompts Used in Code Generation" [2024-04] [paper]

• "Assessing GPT-4-Vision's Capabilities in UML-Based Code Generation" [2024-04] [paper]

• "Large Language Models Synergize with Automated Machine Learning" [2024-05] [paper]

• "Model Cascading for Code: Reducing Inference Costs with Model Cascading for LLM Based Code Generation" [2024-05] [paper]

• "A Survey on Large Language Models for Code Generation" [2024-06] [paper]

• "Is Programming by Example solved by LLMs?" [2024-06] [paper]

• "Benchmarks and Metrics for Evaluations of Code Generation: A Critical Review" [2024-06] [paper]

• "MPCODER: Multi-user Personalized Code Generator with Explicit and Implicit Style Representation Learning" [2024-06] [ACL 2024] [paper]

• "Revisiting the Impact of Pursuing Modularity for Code Generation" [2024-07] [paper]

• "Evaluating Long Range Dependency Handling in Code Generation Models using Multi-Step Key Retrieval" [2024-07] [paper]

• "When to Stop? Towards Efficient Code Generation in LLMs with Excess Token Prevention" [2024-07] [paper]

• "Assessing Programming Task Difficulty for Efficient Evaluation of Large Language Models" [2024-07] [paper]

• "ArchCode: Incorporating Software Requirements in Code Generation with Large Language Models" [2024-08] [ACL 2024] [paper]

• "Fine-tuning Language Models for Joint Rewriting and Completion of Code with Potential Bugs" [2024-08] [ACL 2024 Findings] [paper]

• "Selective Prompt Anchoring for Code Generation" [2024-08] [paper]

• "Bridging the Language Gap: Enhancing Multilingual Prompt-Based Code Generation in LLMs via Zero-Shot Cross-Lingual Transfer" [2024-08] [paper]

• "Optimizing Large Language Model Hyperparameters for Code Generation" [2024-08] [paper]

• "EPiC: Cost-effective Search-based Prompt Engineering of LLMs for Code Generation" [2024-08] [paper]

• "CodeRefine: A Pipeline for Enhancing LLM-Generated Code Implementations of Research Papers" [2024-08] [paper]

• "No Man is an Island: Towards Fully Automatic Programming by Code Search, Code Generation and Program Repair" [2024-09] [paper]

• "Planning In Natural Language Improves LLM Search For Code Generation" [2024-09] [paper]

• "Multi-Programming Language Ensemble for Code Generation in Large Language Model" [2024-09] [paper]

• "A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement" [2024-09] [paper]

• "USCD: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decoding" [2024-09] [paper]

• "Eliciting Instruction-tuned Code Language Models' Capabilities to Utilize Auxiliary Function for Code Generation" [2024-09] [paper]

• "Selection of Prompt Engineering Techniques for Code Generation through Predicting Code Complexity" [2024-09] [paper]

• "Horizon-Length Prediction: Advancing Fill-in-the-Middle Capabilities for Code Generation with Lookahead Planning" [2024-10] [paper]

• "Showing LLM-Generated Code Selectively Based on Confidence of LLMs" [2024-10] [paper]

• "AutoFeedback: An LLM-based Framework for Efficient and Accurate API Request Generation" [2024-10] [paper]

• "Enhancing LLM Agents for Code Generation with Possibility and Pass-rate Prioritized Experience Replay" [2024-10] [paper]

• "From Solitary Directives to Interactive Encouragement! LLM Secure Code Generation by Natural Language Prompting" [2024-10] [paper]

• "Self-Explained Keywords Empower Large Language Models for Code Generation" [2024-10] [paper]

• "Context-Augmented Code Generation Using Programming Knowledge Graphs" [2024-10] [paper]

• "In-Context Code-Text Learning for Bimodal Software Engineering" [2024-10] [paper]

• "Combining LLM Code Generation with Formal Specifications and Reactive Program Synthesis" [2024-10] [paper]

• "Less is More: DocString Compression in Code Generation" [2024-10] [paper]

• "Multi-Programming Language Sandbox for LLMs" [2024-10] [paper]

• "Personality-Guided Code Generation Using Large Language Models" [2024-10] [paper]

• "Do Advanced Language Models Eliminate the Need for Prompt Engineering in Software Engineering?" [2024-11] [paper]

• "Scattered Forest Search: Smarter Code Space Exploration with LLMs" [2024-11] [paper]

• "Anchor Attention, Small Cache: Code Generation with Large Language Models" [2024-11] [paper]

• "ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generation" [2024-11] [paper]

• "SRA-MCTS: Self-driven Reasoning Aurmentation with Monte Carlo Tree Search for Enhanced Code Generation" [2024-11] [paper]

Code RAG

• "CodeGRAG: Extracting Composed Syntax Graphs for Retrieval Augmented Cross-Lingual Code Generation" [2024-05] [paper]

• "Prompt-based Code Completion via Multi-Retrieval Augmented Generation" [2024-05] [paper]

• "A Lightweight Framework for Adaptive Retrieval In Code Completion With Critique Model" [2024-06] [papaer]

• "Preference-Guided Refactored Tuning for Retrieval Augmented Code Generation" [2024-09] [paper]

• "Building A Coding Assistant via the Retrieval-Augmented Language Model" [2024-10] [paper]

• "DroidCoder: Enhanced Android Code Completion with Context-Enriched Retrieval-Augmented Generation" [2024-10] [ASE 2024] [paper]

• "Assessing the Answerability of Queries in Retrieval-Augmented Code Generation" [2024-11] [paper]

Code Ranking

• "Fault-Aware Neural Code Rankers" [2022-06] [NeurIPS 2022] [paper]

• "Functional Overlap Reranking for Neural Code Generation" [2023-10] [ACL 2024 Findings] [paper]

• "Top Pass: Improve Code Generation by Pass@k-Maximized Code Ranking" [2024-08] [paper]

• "DOCE: Finding the Sweet Spot for Execution-Based Code Generation" [2024-08] [paper]

• "Sifting through the Chaff: On Utilizing Execution Feedback for Ranking the Generated Code Candidates" [2024-08] [paper]

• "B4: Towards Optimal Assessment of Plausible Code Solutions with Plausible Tests" [2024-09] [paper]

• "Learning Code Preference via Synthetic Evolution" [2024-10] [paper]

Code Translation

• "Tree-to-tree Neural Networks for Program Translation" [2018-02] [NeurIPS 2018] [paper]

• "Program Language Translation Using a Grammar-Driven Tree-to-Tree Model" [2018-07] [paper]

• "Unsupervised Translation of Programming Languages" [2020-06] [NeurIPS 2020] [paper]

• "Leveraging Automated Unit Tests for Unsupervised Code Translation" [2021-10] [ICLR 2022] paper]

• "Code Translation with Compiler Representations" [2022-06] [ICLR 2023] [paper]

• "Multilingual Code Snippets Training for Program Translation" [2022-06] [AAAI 2022] [paper]

• "BabelTower: Learning to Auto-parallelized Program Translation" [2022-07] [ICML 2022] [paper]

• "Syntax and Domain Aware Model for Unsupervised Program Translation" [2023-02] [ICSE 2023] [paper]

• "CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution" [2023-06] [paper]

• "Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code" [2023-08] [ICSE 2024] [paper]

• "On the Evaluation of Neural Code Translation: Taxonomy and Benchmark", 2023-08, ASE 2023, [paper]

• "Program Translation via Code Distillation" [2023-10] [EMNLP 2023] [paper]

• "Explain-then-Translate: An Analysis on Improving Program Translation with Self-generated Explanations" [2023-11] [EMNLP 2023 Findings] [paper]

• "Exploring the Impact of the Output Format on the Evaluation of Large Language Models for Code Translation" [2024-03] [paper]

• "Exploring and Unleashing the Power of Large Language Models in Automated Code Translation" [2024-04] [paper]

• "VERT: Verified Equivalent Rust Transpilation with Few-Shot Learning" [2024-04] [paper]

• "Towards Translating Real-World Code with LLMs: A Study of Translating to Rust" [2024-05] [paper]

• "An interpretable error correction method for enhancing code-to-code translation" [2024-05] [ICLR 2024] [paper]

• "LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes" [2024-06] [paper]

• "Rectifier: Code Translation with Corrector via LLMs" [2024-07] [paper]

• "Enhancing Code Translation in Language Models with Few-Shot Learning via Retrieval-Augmented Generation" [2024-07] [paper]

• "A Joint Learning Model with Variational Interaction for Multilingual Program Translation" [2024-08] [paper]

• "Automatic Library Migration Using Large Language Models: First Results" [2024-08] [paper]

• "Context-aware Code Segmentation for C-to-Rust Translation using Large Language Models" [2024-09] [paper]

• "TRANSAGENT: An LLM-Based Multi-Agent System for Code Translation" [2024-10] [paper]

• "Unraveling the Potential of Large Language Models in Code Translation: How Far Are We?" [2024-10] [paper]

• "CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming" [2024-10] [paper]

• "A test-free semantic mistakes localization framework in Neural Code Translation" [2024-10] [paper]

• "Repository-Level Compositional Code Translation and Validation" [2024-10] [paper]

• "Leveraging Large Language Models for Code Translation and Software Development in Scientific Computing" [2024-10] [paper]

• "InterTrans: Leveraging Transitive Intermediate Translations to Enhance LLM-based Code Translation" [2024-11] [paper]

• "Translating C To Rust: Lessons from a User Study" [2024-11] [paper]

Code Commenting and Summarization

• "A Transformer-based Approach for Source Code Summarization" [2020-05] [ACL 2020] [paper]

• "Code Summarization with Structure-induced Transformer" [2020-12] [ACL 2021 Findings] [paper]

• "Code Structure Guided Transformer for Source Code Summarization" [2021-04] [ACM TSEM] [paper]

• "M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization" [2022-03] [ICPC 2022] [paper]

• "AST-trans: code summarization with efficient tree-structured attention" [2022-05] [ICSE 2022] [paper]

• "CoSS: Leveraging Statement Semantics for Code Summarization" [2023-03] [IEEE TSE] [paper]

• "Automatic Code Summarization via ChatGPT: How Far Are We?" [2023-05] [paper]

• "Semantic Similarity Loss for Neural Source Code Summarization" [2023-08] [paper]

• "Distilled GPT for Source Code Summarization" [2023-08] [ASE] [paper]

• "CSA-Trans: Code Structure Aware Transformer for AST" [2024-04] [paper]

• "Analyzing the Performance of Large Language Models on Code Summarization" [2024-04] [paper]

• "Enhancing Trust in LLM-Generated Code Summaries with Calibrated Confidence Scores" [2024-04] [paper]

• "DocuMint: Docstring Generation for Python using Small Language Models" [2024-05] [paper] [repo]

• "Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large Language Models" [2024-05] [paper]

• "Large Language Models for Code Summarization" [2024-05] [paper]

• "Exploring the Efficacy of Large Language Models (GPT-4) in Binary Reverse Engineering" [2024-06] [paper]

• "Identifying Inaccurate Descriptions in LLM-generated Code Comments via Test Execution" [2024-06] [paper]

• "MALSIGHT: Exploring Malicious Source Code and Benign Pseudocode for Iterative Binary Malware Summarization" [2024-06] [paper]

• "ESALE: Enhancing Code-Summary Alignment Learning for Source Code Summarization" [2024-07] [paper]

• "Source Code Summarization in the Era of Large Language Models" [2024-07] [paper]

• "Natural Language Outlines for Code: Literate Programming in the LLM Era" [2024-08] [paper]

• "Context-aware Code Summary Generation" [2024-08] [paper]

• "AUTOGENICS: Automated Generation of Context-Aware Inline Comments for Code Snippets on Programming Q&A Sites Using LLM" [2024-08] [paper]

• "LLMs as Evaluators: A Novel Approach to Evaluate Bug Report Summarization" [2024-09] [paper]

• "Evaluating the Quality of Code Comments Generated by Large Language Models for Novice Programmers" [2024-09] [paper]

• "Generating Equivalent Representations of Code By A Self-Reflection Approach" [2024-10] [paper]

• "A review of automatic source code summarization" [2024-10] [Empirical Software Engineering] [paper]

Program Repair

• "DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletons" [2021-05] [paper]

• "Break-It-Fix-It: Unsupervised Learning for Program Repair" [2021-06] [ICML 2021] [paper]

• "TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer" [2021-07] [ICML 2021] [paper]

• "Automated Repair of Programs from Large Language Models" [2022-05] [ICSE 2023] [paper]

• "Less Training, More Repairing Please: Revisiting Automated Program Repair via Zero-shot Learning" [2022-07] [ESEC/FSE 2022] [paper]

• "Repair Is Nearly Generation: Multilingual Program Repair with LLMs" [2022-08] [AAAI 2023] [paper]

• "Practical Program Repair in the Era of Large Pre-trained Language Models" [2022-10] [paper]

• "VulRepair: a T5-based automated software vulnerability repair" [2022-11] [ESEC/FSE 2022] [paper]

• "Conversational Automated Program Repair" [2023-01] [paper]

• "Impact of Code Language Models on Automated Program Repair" [2023-02] [ICSE 2023] [paper]

• "InferFix: End-to-End Program Repair with LLMs" [2023-03] [ESEC/FSE 2023] [paper]

• "Enhancing Automated Program Repair through Fine-tuning and Prompt Engineering" [2023-04] [paper]

• "A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair" [2023-04] [paper]

• "Domain Knowledge Matters: Improving Prompts with Fix Templates for Repairing Python Type Errors" [2023-06] [ICSE 2024] [paper]

• "RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair" [2023-12] [paper]

• "The Fact Selection Problem in LLM-Based Program Repair" [2024-04] [paper]

• "Aligning LLMs for FL-free Program Repair" [2024-04] [paper]

• "A Deep Dive into Large Language Models for Automated Bug Localization and Repair" [2024-04] [paper]

• "Multi-Objective Fine-Tuning for Enhanced Program Repair with LLMs" [2024-04] [paper]

• "How Far Can We Go with Practical Function-Level Program Repair?" [2024-04] [paper]

• "Revisiting Unnaturalness for Automated Program Repair in the Era of Large Language Models" [2024-04] [paper]

• "A Unified Debugging Approach via LLM-Based Multi-Agent Synergy" [2024-04] [paper]

• "A Systematic Literature Review on Large Language Models for Automated Program Repair" [2024-05] [paper]

• "NAVRepair: Node-type Aware C/C++ Code Vulnerability Repair" [2024-05] [paper]

• "Automated Program Repair: Emerging trends pose and expose problems for benchmarks" [2024-05] [paper]

• "Automated Repair of AI Code with Large Language Models and Formal Verification" [2024-05] [paper]

• "A Case Study of LLM for Automated Vulnerability Repair: Assessing Impact of Reasoning and Patch Validation Feedback" [2024-05] [paper]

• "CREF: An LLM-based Conversational Software Repair Framework for Programming Tutors" [2024-06] [paper]

• "Towards Practical and Useful Automated Program Repair for Debugging" [2024-07] [paper]

• "ThinkRepair: Self-Directed Automated Program Repair" [2024-07] [paper]

• "MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair" [2024-08] [paper]

• "RePair: Automated Program Repair with Process-based Feedback" [2024-08] [ACL 2024 Findings] [paper]

• "Enhancing LLM-Based Automated Program Repair with Design Rationales" [2024-08] [paper]

• "Automated Software Vulnerability Patching using Large Language Models" [2024-08] [paper]

• "Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairs" [2024-09] [paper]

• "MarsCode Agent: AI-native Automated Bug Fixing" [2024-09] [paper]

• "Co-Learning: Code Learning for Multi-Agent Reinforcement Collaborative Framework with Conversational Natural Language Interfaces" [2024-09] [paper]

• "Debugging with Open-Source Large Language Models: An Evaluation" [2024-09] [paper]

• "VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patching" [2024-09] [paper]

• "ContractTinker: LLM-Empowered Vulnerability Repair for Real-World Smart Contracts" [2024-09] [paper]

• "Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs" [2024-09] [paper]

• "Exploring and Lifting the Robustness of LLM-powered Automated Program Repair with Metamorphic Testing" [2024-10] [paper]

• "LecPrompt: A Prompt-based Approach for Logical Error Correction with CodeBERT" [2024-10] [paper]

• "Semantic-guided Search for Efficient Program Repair with Large Language Models" [2024-10] [paper]

• "A Comprehensive Survey of AI-Driven Advancements and Techniques in Automated Program Repair and Code Generation" [2024-11] [paper]

Code Similarity and Embedding (Clone Detection, Code Search)

• "Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations" [2020-09] [SIGIR 2021] [paper]

• "REINFOREST: Reinforcing Semantic Code Similarity for Cross-Lingual Code Search Models" [2023-05] [paper]

• "Rewriting the Code: A Simple Method for Large Language Model Augmented Code Search" [2024-01] [ACL 2024] [paper]

• "Revisiting Code Similarity Evaluation with Abstract Syntax Tree Edit Distance" [2024-04] [ACL 2024 short] [paper]

• "Is Next Token Prediction Sufficient for GPT? Exploration on Code Logic Comprehension" [2024-04] [paper]

• "Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuning" [2024-05] [paper]

• "Typhon: Automatic Recommendation of Relevant Code Cells in Jupyter Notebooks" [2024-05] [paper]

• "Toward Exploring the Code Understanding Capabilities of Pre-trained Code Generation Models" [2024-06] [paper]

• "Aligning Programming Language and Natural Language: Exploring Design Choices in Multi-Modal Transformer-Based Embedding for Bug Localization" [2024-06] [paper]

• "Assessing the Code Clone Detection Capability of Large Language Models" [2024-07] [paper]

• "CodeCSE: A Simple Multilingual Model for Code and Comment Sentence Embeddings" [2024-07] [paper]

• "Large Language Models for cross-language code clone detection" [2024-08] [paper]

• "Coding-PTMs: How to Find Optimal Code Pre-trained Models for Code Embedding in Vulnerability Detection?" [2024-08] [paper]

• "You Augment Me: Exploring ChatGPT-based Data Augmentation for Semantic Code Search" [2024-08] [paper]

• "Improving Source Code Similarity Detection Through GraphCodeBERT and Integration of Additional Features" [2024-08] [paper]

• "LLM Agents Improve Semantic Code Search" [2024-08] [paper]

• "zsLLMCode: An Effective Approach for Functional Code Embedding via LLM with Zero-Shot Learning" [2024-09] [paper]

• "Exploring Demonstration Retrievers in RAG for Coding Tasks: Yeas and Nays!" [2024-10] [paper]

• "Instructive Code Retriever: Learn from Large Language Model's Feedback for Code Intelligence Tasks" [2024-10] [paper]

• "Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations" [2024-10] [paper]

• "Are Decoder-Only Large Language Models the Silver Bullet for Code Search?" [2024-10] [paper]

• "CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval" [2024-11] [paper]

• "CodeSAM: Source Code Representation Learning by Infusing Self-Attention with Multi-Code-View Graphs" [2024-11] [paper]

• "EnStack: An Ensemble Stacking Framework of Large Language Models for Enhanced Vulnerability Detection in Source Code" [2024-11] [paper]

• "Isotropy Matters: Soft-ZCA Whitening of Embeddings for Semantic Code Search" [2024-11] [paper]

Code Refactoring

• "An Empirical Study on the Code Refactoring Capability of Large Language Models" [2024-11] [paper]

• "Automated Update of Android Deprecated API Usages with Large Language Models" [2024-11] [paper]

• "An Empirical Study on the Potential of LLMs in Automated Software Refactoring" [2024-11] [paper]

• "CODECLEANER: Elevating Standards with A Robust Data Contamination Mitigation Toolkit" [2024-11] [paper]

• "Instruct or Interact? Exploring and Eliciting LLMs' Capability in Code Snippet Adaptation Through Prompt Engineering" [2024-11] [paper]

Type Prediction

• "Learning type annotation: is big data enough?" [2021-08] [ESEC/FSE 2021] [paper]

• "Do Machine Learning Models Produce TypeScript Types That Type Check?" [2023-02] [ECOOP 2023] [paper]

• "TypeT5: Seq2seq Type Inference using Static Analysis" [2023-03] [ICLR 2023] [paper]

• "Type Prediction With Program Decomposition and Fill-in-the-Type Training" [2023-05] [paper]

• "Generative Type Inference for Python" [2023-07] [ASE 2023] [paper]

• "Activation Steering for Robust Type Prediction in CodeLLMs" [2024-04] [paper]

• "An Empirical Study of Large Language Models for Type and Call Graph Analysis" [2024-10] [paper]

Repository-Level Coding

• "Repository-Level Prompt Generation for Large Language Models of Code" [2022-06] [ICML 2023] [paper]

• "CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context" [2022-12] [paper]

• "RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation" [2023-03] [EMNLP 2023] [paper]

• "Coeditor: Leveraging Repo-level Diffs for Code Auto-editing" [2023-05] [ICLR 2024 Spotlight] [paper]

• "RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems" [2023-06] [ICLR 2024] [paper]

• "Guiding Language Models of Code with Global Context using Monitors" [2023-06] [paper]

• "RepoFusion: Training Code Models to Understand Your Repository" [2023-06] [paper]

• "CodePlan: Repository-level Coding using LLMs and Planning" [2023-09] [paper]

• "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?" [2023-10] [ICLR 2024] [paper]

• "CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion" [2023-10] [NeurIPS 2023] [paper]

• "A^3-CodGen: A Repository-Level Code Generation Framework for Code Reuse with Local-Aware, Global-Aware, and Third-Party-Library-Aware" [2023-12] [paper]

• "Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation" [2024-01] [paper]

• "RepoHyper: Better Context Retrieval Is All You Need for Repository-Level Code Completion" [2024-03] [paper]

• "Repoformer: Selective Retrieval for Repository-Level Code Completion" [2024-03] [ICML 2024] [paper]

• "CodeS: Natural Language to Code Repository via Multi-Layer Sketch" [2024-03] [paper]

• "Class-Level Code Generation from Natural Language Using Iterative, Tool-Enhanced Reasoning over Repository" [2024-04] [paper]

• "Contextual API Completion for Unseen Repositories Using LLMs" [2024-05] [paper]

• "Dataflow-Guided Retrieval Augmentation for Repository-Level Code Completion" [2024-05][ACL 2024] [paper]

• "How to Understand Whole Software Repository?" [2024-06] [paper]

• "R2C2-Coder: Enhancing and Benchmarking Real-world Repository-level Code Completion Abilities of Code Large Language Models" [2024-06] [paper]

• "CodeR: Issue Resolving with Multi-Agent and Task Graphs" [2024-06] [paper]

• "Enhancing Repository-Level Code Generation with Integrated Contextual Information" [2024-06] [paper]

• "On The Importance of Reasoning for Context Retrieval in Repository-Level Code Editing" [2024-06] [paper]

• "GraphCoder: Enhancing Repository-Level Code Completion via Code Context Graph-based Retrieval and Language Model" [2024-06] [ASE 2024] [paper]

• "STALL+: Boosting LLM-based Repository-level Code Completion with Static Analysis" [2024-06] [paper]

• "Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMs" [2024-06] [paper]

• "Agentless: Demystifying LLM-based Software Engineering Agents" [2024-07] [paper]

• "RLCoder: Reinforcement Learning for Repository-Level Code Completion" [2024-07] [paper]

• "CoEdPilot: Recommending Code Edits with Learned Prior Edit Relevance, Project-wise Awareness, and Interactive Nature" [2024-08] [paper] [repo]

• "RAMBO: Enhancing RAG-based Repository-Level Method Body Completion" [2024-09] [paper]

• "Exploring the Potential of Conversational Test Suite Based Program Repair on SWE-bench" [2024-10] [paper]

• "RepoGraph: Enhancing AI Software Engineering with Repository-level Code Graph" [2024-10] [paper]

• "See-Saw Generative Mechanism for Scalable Recursive Code Generation with Generative AI" [2024-11] [paper]

Frontend Development

• "Seeking the user interface", 2014-09, ASE 2014, [paper]

• "pix2code: Generating Code from a Graphical User Interface Screenshot", 2017-05, EICS 2018, [paper]

• "Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps", 2018-02, TSE 2020, [paper]

• "Automatic HTML Code Generation from Mock-Up Images Using Machine Learning Techniques", 2019-04, EBBT 2019, [paper]

• "Sketch2code: Generating a website from a paper mockup", 2019-05, [paper]

• "HTLM: Hyper-Text Pre-Training and Prompting of Language Models", 2021-07, ICLR 2022, [paper]

• "Learning UI-to-Code Reverse Generator Using Visual Critic Without Rendering", 2023-05, [paper]

• "Design2Code: How Far Are We From Automating Front-End Engineering?" [2024-03] [paper]

• "Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset" [2024-03] [paper]

• "VISION2UI: A Real-World Dataset with Layout for Code Generation from UI Designs" [2024-04] [paper]

• "LogoMotion: Visually Grounded Code Generation for Content-Aware Animation" [2024-05] [paper]

• "PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM" [2024-06] [paper]

• "UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback" [2024-06] [paper]

• "On AI-Inspired UI-Design" [2024-06] [paper]

• "Identifying User Goals from UI Trajectories" [2024-06] [paper]

• "Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approach" [2024-06] [paper]

• "Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs" [2024-06] [paper]

• "Vision-driven Automated Mobile GUI Testing via Multimodal Large Language Model" [2024-07] [paper]

• "AUITestAgent: Automatic Requirements Oriented GUI Function Testing" [2024-07] [paper]

• "LLM-based Abstraction and Concretization for GUI Test Migration" [2024-09] [paper]

• "Enabling Cost-Effective UI Automation Testing with Retrieval-Based LLMs: A Case Study in WeChat" [2024-09] [paper]

• "Self-Elicitation of Requirements with Automated GUI Prototyping" [2024-09] [paper]

• "Infering Alt-text For UI Icons With Large Language Models During App Development" [2024-09] [paper]

• "Leveraging Large Vision Language Model For Better Automatic Web GUI Testing" [2024-10] [paper]

• "Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping" [2024-10] [paper]

• "WAFFLE: Multi-Modal Model for Automated Front-End Development" [2024-10] [paper]

• "DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models" [2024-11] [paper]

• "Interaction2Code: How Far Are We From Automatic Interactive Webpage Generation?" [2024-11] [paper]

• "A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs" [2024-11] [paper]

Text-To-SQL

• "PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models" [2021-09] [EMNLP 2021] [paper]

• "CodexDB: Generating Code for Processing SQL Queries using GPT-3 Codex" [2022-04] [paper]

• "T5QL: Taming language models for SQL generation" [2022-09] [paper]

• "Towards Generalizable and Robust Text-to-SQL Parsing" [2022-10] [EMNLP 2022 Findings] [paper]

• "XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing" [2022-10] [EMNLP 2022 Findings] [paper]

• "A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability" [2023-03] [paper]

• "DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction" [2023-04] [NeurIPS 2023] [paper]

• "How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings" [2023-05] [paper]

• "Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies" [2023-05] [paper]

• "SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL" [2023-05] [paper]

• "Retrieval-augmented GPT-3.5-based Text-to-SQL Framework with Sample-aware Prompting and Dynamic Revision Chain" [2023-07] [ICONIP 2023] [paper]

• "Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation" [2023-08] [paper]

• "MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL" [2023-12] [paper]

• "Investigating the Impact of Data Contamination of Large Language Models in Text-to-SQL Translation" [2024-02] [ACL 2024 Findings] [paper]

• "Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm" [2024-02] [ACL 2024 Findings] [paper]

• "Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM" [2024-02] [ACL 2024 Findings] [paper]

• "Understanding the Effects of Noise in Text-to-SQL: An Examination of the BIRD-Bench Benchmark" [2024-02] [ACL 2024 short] [paper]

• "SQL-Encoder: Improving NL2SQL In-Context Learning Through a Context-Aware Encoder" [2024-03] [paper]

• "LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency" [2024-04] [paper]

• "Dubo-SQL: Diverse Retrieval-Augmented Generation and Fine Tuning for Text-to-SQL" [2024-04] [paper]

• "EPI-SQL: Enhancing Text-to-SQL Translation with Error-Prevention Instructions" [2024-04] [paper]

• "ProbGate at EHRSQL 2024: Enhancing SQL Query Generation Accuracy through Probabilistic Threshold Filtering and Error Handling" [2024-04] [paper]

• "CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions" [2024-05] [paper]

• "Open-SQL Framework: Enhancing Text-to-SQL on Open-source Large Language Models" [2024-05] [paper]

• "MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation" [2024-05] [paper]

• "PromptMind Team at EHRSQL-2024: Improving Reliability of SQL Generation using Ensemble LLMs" [2024-05] [paper]

• "LG AI Research & KAIST at EHRSQL 2024: Self-Training Large Language Models with Pseudo-Labeled Unanswerable Questions for a Reliable Text-to-SQL System on EHRs" [2024-05] [paper]

• "Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generation" [2024-05] [ACL 2024 Findings] [paper]

• "CHESS: Contextual Harnessing for Efficient SQL Synthesis" [2024-05] [paper]

• "DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning" [2024-06] [paper]

• "Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL" [2024-06] [paper]

• "RH-SQL: Refined Schema and Hardness Prompt for Text-to-SQL" [2024-06] [paper]

• "QDA-SQL: Questions Enhanced Dialogue Augmentation for Multi-Turn Text-to-SQL" [2024-06] [paper]

• "End-to-end Text-to-SQL Generation within an Analytics Insight Engine" [2024-06] [paper]

• "MAGIC: Generating Self-Correction Guideline for In-Context Text-to-SQL" [2024-06] [paper]

• "SQLFixAgent: Towards Semantic-Accurate SQL Generation via Multi-Agent Collaboration" [2024-06] [paper]

• "Unmasking Database Vulnerabilities: Zero-Knowledge Schema Inference Attacks in Text-to-SQL Systems" [2024-06] [paper]

• "Lucy: Think and Reason to Solve Text-to-SQL" [2024-07] [paper]

• "ESM+: Modern Insights into Perspective on Text-to-SQL Evaluation in the Age of Large Language Models" [2024-07] [paper]

• "RB-SQL: A Retrieval-based LLM Framework for Text-to-SQL" [2024-07] [paper]

• "AI-Assisted SQL Authoring at Industry Scale" [2024-07] [paper]

• "SQLfuse: Enhancing Text-to-SQL Performance through Comprehensive LLM Synergy" [2024-07] [paper]

• "A Survey on Employing Large Language Models for Text-to-SQL Tasks" [2024-07] [paper]

• "Towards Automated Data Sciences with Natural Language and SageCopilot: Practices and Lessons Learned" [2024-07] [paper]

• "Evaluating LLMs for Text-to-SQL Generation With Complex SQL Workload" [2024-07] [paper]

• "Synthesizing Text-to-SQL Data from Weak and Strong LLMs" [2024-08] [ACL 2024] [paper]

• "Improving Relational Database Interactions with Large Language Models: Column Descriptions and Their Impact on Text-to-SQL Performance" [2024-08] [paper]

• "The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models" [2024-08] [paper]

• "MAG-SQL: Multi-Agent Generative Approach with Soft Schema Linking and Iterative Sub-SQL Refinement for Text-to-SQL" [2024-08] [paper]

• "Enhancing Text-to-SQL Parsing through Question Rewriting and Execution-Guided Refinement" [2024-08] [ACL 2024 Findings] [paper]

• "DAC: Decomposed Automation Correction for Text-to-SQL" [2024-08] [paper]

• "Interactive-T2S: Multi-Turn Interactions for Text-to-SQL with Large Language Models" [2024-08] [paper]

• "SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging" [2024-08] [paper]

• "Enhancing SQL Query Generation with Neurosymbolic Reasoning" [2024-08] [paper]

• "Text2SQL is Not Enough: Unifying AI and Databases with TAG" [2024-08] [paper]

• "Tool-Assisted Agent on SQL Inspection and Refinement in Real-World Scenarios" [2024-08] [paper]

• "SelECT-SQL: Self-correcting ensemble Chain-of-Thought for Text-to-SQL" [2024-09] [paper]

• "You Only Read Once (YORO): Learning to Internalize Database Knowledge for Text-to-SQL" [2024-09] [paper]

• "PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL" [2024-09] [paper]

• "Enhancing Text-to-SQL Capabilities of Large Language Models via Domain Database Knowledge Injection" [2024-09] [paper]

• "DataGpt-SQL-7B: An Open-Source Language Model for Text-to-SQL" [2024-09] [paper]

• "E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL" [2024-09] [paper]

• "FLEX: Expert-level False-Less EXecution Metric for Reliable Text-to-SQL Benchmark" [2024-09] [paper]

• "Enhancing LLM Fine-tuning for Text-to-SQLs by SQL Quality Measurement" [2024-10] [paper]

• "From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems" [2024-10] [paper]

• "CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL" [2024-10] [paper]

• "Context-Aware SQL Error Correction Using Few-Shot Learning -- A Novel Approach Based on NLQ, Error, and SQL Similarity" [2024-10] [paper]

• "Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL" [2024-10] [paper]

• "LR-SQL: A Supervised Fine-Tuning Method for Text2SQL Tasks under Low-Resource Scenarios" [2024-10] [paper]

• "MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation" [2024-10] [paper]

• "Learning Metadata-Agnostic Representations for Text-to-SQL In-Context Example Selection" [2024-10] [paper]

• "An Actor-Critic Approach to Boosting Text-to-SQL Large Language Model" [2024-10] [paper]

• "RSL-SQL: Robust Schema Linking in Text-to-SQL Generation" [2024-10] [paper]

• "KeyInst: Keyword Instruction for Improving SQL Formulation in Text-to-SQL" [2024-10] [paper]

• "Grounding Natural Language to SQL Translation with Data-Based Self-Explanations" [2024-11] [paper]

• "PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing" [2024-11] [paper]

• "XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL" [2024-11] [paper]

• "Leveraging Prior Experience: An Expandable Auxiliary Knowledge Base for Text-to-SQL" [2024-11] [paper]

• "Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities" [2024-11] [paper]

Program Proof

• "Baldur: Whole-Proof Generation and Repair with Large Language Models" [2023-03] [FSE 2023] [paper]

• "An In-Context Learning Agent for Formal Theorem-Proving" [2023-10] [paper]

• "Towards AI-Assisted Synthesis of Verified Dafny Methods" [2024-02] [FSE 2024] [paper]

• "Towards Neural Synthesis for SMT-Assisted Proof-Oriented Programming" [2024-05] [paper]

• "Laurel: Generating Dafny Assertions Using Large Language Models" [2024-05] [paper]

• "AutoVerus: Automated Proof Generation for Rust Code" [2024-09] [paper]

• "Proof Automation with Large Language Models" [2024-09] [paper]

• "Automated Proof Generation for Rust Code via Self-Evolution" [2024-10] [paper]

• "CoqPilot, a plugin for LLM-based generation of proofs" [2024-10] [paper]

• "dafny-annotator: AI-Assisted Verification of Dafny Programs" [2024-11] [paper]

Test Generation

• "Unit Test Case Generation with Transformers and Focal Context" [2020-09] [AST@ICSE 2022] [paper]

• "An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation" [2023-02] [IEEE TSE] [paper]

• "A3Test: Assertion-Augmented Automated Test Case Generation" [2023-02] [paper]

• "Learning Deep Semantics for Test Completion" [2023-02] [ICSE 2023] [paper]

• "Using Large Language Models to Generate JUnit Tests: An Empirical Study" [2023-04] [EASE 2024] [paper]

• "CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-Trained Large Language Models" [2023-05] [ICSE 2023] [paper]

• "No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation" [2023-05] [paper]

• "ChatUniTest: a ChatGPT-based automated unit test generation tool" [2023-05] [paper]

• "ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation" [2023-07] [paper]

• "Can Large Language Models Write Good Property-Based Tests?" [2023-07] [paper]

• "Domain Adaptation for Deep Unit Test Case Generation" [2023-08] [paper]

• "Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing" [2023-08] [paper]

• "How well does LLM generate security tests?" [2023-10] [paper]

• "Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation" [2023-10] [paper]

• "An initial investigation of ChatGPT unit test generation capability" [2023-10] [SAST 2023] [paper]

• "CoverUp: Coverage-Guided LLM-Based Test Generation" [2024-03] [paper]

• "Enhancing LLM-based Test Generation for Hard-to-Cover Branches via Program Analysis" [2024-04] [paper]

• "Large Language Models for Mobile GUI Text Input Generation: An Empirical Study" [2024-04] [paper]

• "Test Code Generation for Telecom Software Systems using Two-Stage Generative Model" [2024-04] [paper]

• "LLM-Powered Test Case Generation for Detecting Tricky Bugs" [2024-04] [paper]

• "Generating Test Scenarios from NL Requirements using Retrieval-Augmented LLMs: An Industrial Study" [2024-04] [paper]

• "Large Language Models as Test Case Generators: Performance Evaluation and Enhancement" [2024-04] [paper]

• "Leveraging Large Language Models for Automated Web-Form-Test Generation: An Empirical Study" [2024-05] [paper]

• "DLLens: Testing Deep Learning Libraries via LLM-aided Synthesis" [2024-06] [paper]

• "Exploring Fuzzing as Data Augmentation for Neural Test Generation" [2024-06] [paper]

• "Mokav: Execution-driven Differential Testing with LLMs" [2024-06] [paper]

• "Code Agents are State of the Art Software Testers" [2024-06] [paper]

• "CasModaTest: A Cascaded and Model-agnostic Self-directed Framework for Unit Test Generation" [2024-06] [paper]

• "An Empirical Study of Unit Test Generation with Large Language Models" [2024-06] [paper]

• "Large-scale, Independent and Comprehensive study of the power of LLMs for test case generation" [2024-06] [paper]

• "Augmenting LLMs to Repair Obsolete Test Cases with Static Collector and Neural Reranker" [2024-07] [paper]

• "Harnessing the Power of LLMs: Automating Unit Test Generation for High-Performance Computing" [2024-07] [paper]

• "An LLM-based Readability Measurement for Unit Tests' Context-aware Inputs" [2024-07] [paper]

• "A System for Automated Unit Test Generation Using Large Language Models and Assessment of Generated Test Suites" [2024-08] [paper]

• "Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests" [2024-08] [paper]

• "Multi-language Unit Test Generation using LLMs" [2024-09] [paper]

• "Exploring the Integration of Large Language Models in Industrial Test Maintenance Processes" [2024-09] [paper]

• "Python Symbolic Execution with LLM-powered Code Generation" [2024-09] [paper]

• "Rethinking the Influence of Source Code on Test Case Generation" [2024-09] [paper]

• "On the Effectiveness of LLMs for Manual Test Verifications" [2024-09] [paper]

• "Retrieval-Augmented Test Generation: How Far Are We?" [2024-09] [paper]

• "Context-Enhanced LLM-Based Framework for Automatic Test Refactoring" [2024-09] [paper]

• "TestBench: Evaluating Class-Level Test Case Generation Capability of Large Language Models" [2024-09] [paper]

• "Advancing Bug Detection in Fastjson2 with Large Language Models Driven Unit Test Generation" [2024-10] [paper]

• "Test smells in LLM-Generated Unit Tests" [2024-10] [paper]

• "LLM-based Unit Test Generation via Property Retrieval" [2024-10] [paper]

• "Disrupting Test Development with AI Assistants" [2024-11] [paper]

• "Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Study" [2024-11] [paper]

• "VALTEST: Automated Validation of Language Model Generated Test Cases" [2024-11] [paper]

• "REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models" [2024-11] [paper]

Oracle Generation

• "Generating Accurate Assert Statements for Unit Test Cases using Pretrained Transformers" [2020-09] [paper]

• "TOGA: A Neural Method for Test Oracle Generation" [2021-09] [ICSE 2022] [paper]

• "TOGLL: Correct and Strong Test Oracle Generation with LLMs" [2024-05] [paper]

• "Test Oracle Automation in the era of LLMs" [2024-05] [paper]

• "Beyond Code Generation: Assessing Code LLM Maturity with Postconditions" [2024-07] [paper]

• "Chat-like Asserts Prediction with the Support of Large Language Model" [2024-07] [paper]

• "Do LLMs generate test oracles that capture the actual or the expected program behaviour?" [2024-10] [paper]

• "Generating executable oracles to check conformance of client code to requirements of JDK Javadocs using LLMs" [2024-11] [paper]

• "Automatically Write Code Checker: An LLM-based Approach with Logic-guided API Retrieval and Case by Case Iteration" [2024-11] [paper]

• "ASSERTIFY: Utilizing Large Language Models to Generate Assertions for Production Code" [2024-11] [paper]

Mutation Testing

• "μBERT: Mutation Testing using Pre-Trained Language Models" [2022-03] [paper]

• "Efficient Mutation Testing via Pre-Trained Language Models" [2023-01] [paper]

• "LLMorpheus: Mutation Testing using Large Language Models" [2024-04] [paper]

• "An Exploratory Study on Using Large Language Models for Mutation Testing" [2024-06] [paper]

• "Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats" [2024-10] [paper]

Fuzz Testing

• "Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models" [2022-12] [paper]

• "Fuzz4All: Universal Fuzzing with Large Language Models" [2023-08] [paper]

• "WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models" [2023-10] [paper]

• "LLAMAFUZZ: Large Language Model Enhanced Greybox Fuzzing" [2024-06] [paper]

• "FuzzCoder: Byte-level Fuzzing Test via Large Language Model" [2024-09] [paper]

• "ISC4DGF: Enhancing Directed Grey-box Fuzzing with LLM-Driven Initial Seed Corpus Generation" [2024-09] [paper]

• "Large Language Models Based JSON Parser Fuzzing for Bug Discovery and Behavioral Analysis" [2024-10] [paper]

• "Fixing Security Vulnerabilities with AI in OSS-Fuzz" [2024-11] [paper]

• "A Code Knowledge Graph-Enhanced System for LLM-Based Fuzz Driver Generation" [2024-11] [paper]

Vulnerability Detection

• "VulDeePecker: A Deep Learning-Based System for Vulnerability Detection" [2018-01] [NDSS 2018] [paper]

• "DeepBugs: A Learning Approach to Name-based Bug Detection" [2018-04] [Proc. ACM Program. Lang.] [paper]

• "Automated Vulnerability Detection in Source Code Using Deep Representation Learning" [2018-07] [ICMLA 2018] [paper]

• "SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities" [2018-07] [IEEE TDSC] [paper]

• "Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks" [2019-09] [NeurIPS 2019] [paper]

• "Improving bug detection via context-based code representation learning and attention-based neural networks" [2019-10] [Proc. ACM Program. Lang.] [paper]

• "Global Relational Models of Source Code" [2019-12] [ICLR 2020] [paper]

• "VulDeeLocator: A Deep Learning-based Fine-grained Vulnerability Detector" [2020-01] [IEEE TDSC] [paper]

• "Deep Learning based Vulnerability Detection: Are We There Yet?" [2020-09] [IEEE TSE] [paper]

• "Security Vulnerability Detection Using Deep Learning Natural Language Processing" [2021-05] [INFOCOM Workshops 2021] [paper]

• "Self-Supervised Bug Detection and Repair" [2021-05] [NeurIPS 2021] [paper]

• "Vulnerability Detection with Fine-grained Interpretations" [2021-06] [ESEC/SIGSOFT FSE 2021] [paper]

• "ReGVD: Revisiting Graph Neural Networks for Vulnerability Detection" [2021-10] [ICSE Companion 2022] [paper]

• "VUDENC: Vulnerability Detection with Deep Learning on a Natural Codebase for Python" [2022-01] [Inf. Softw. Technol] [paper]

• "Transformer-Based Language Models for Software Vulnerability Detection" [222-04] [ACSAC 2022] [paper]

• "LineVul: A Transformer-based Line-Level Vulnerability Prediction" [2022-05] [MSR 2022] [paper]

• "VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection" [2022-05] [IJCNN 2022] [paper]

• "Open Science in Software Engineering: A Study on Deep Learning-Based Vulnerability Detection" [2022-09] [IEEE TSE] [paper]

• "An Empirical Study of Deep Learning Models for Vulnerability Detection" [2022-12] [ICSE 2023] [paper]

• "CSGVD: A deep learning approach combining sequence and graph embedding for source code vulnerability detection" [2023-01] [J. Syst. Softw.] [paper]

• "Benchmarking Software Vulnerability Detection Techniques: A Survey" [2023-03] [paper]

• "Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?" [2023-05] [paper]

• "A Survey on Automated Software Vulnerability Detection Using Machine Learning and Deep Learning" [2023-06] [paper]

• "Limits of Machine Learning for Automatic Vulnerability Detection" [2023-06] [paper]

• "Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation" [2023-08] [paper]

• "Prompt-Enhanced Software Vulnerability Detection Using ChatGPT" [2023-08] [paper]

• "Towards Causal Deep Learning for Vulnerability Detection" [2023-10] [paper]

• "Understanding the Effectiveness of Large Language Models in Detecting Security Vulnerabilities" [2023-11] [paper]

• "How Far Have We Gone in Vulnerability Detection Using Large Language Models" [2023-11] [paper]

• "Can Large Language Models Identify And Reason About Security Vulnerabilities? Not Yet" [2023-12] [paper]

• "LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability Reasoning" [2024-01] [paper]

• "Security Code Review by LLMs: A Deep Dive into Responses" [2024-01] [paper]

• "Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilities" [2024-02] [paper]

• "Multi-role Consensus through LLMs Discussions for Vulnerability Detection" [2024-03] [paper]

• "A Comprehensive Study of the Capabilities of Large Language Models for Vulnerability Detection" [2024-03] [paper]

• "Vulnerability Detection with Code Language Models: How Far Are We?" [2024-03] [paper]

• "Multitask-based Evaluation of Open-Source LLM on Software Vulnerability" [2024-04] [paper]

• "Large Language Model for Vulnerability Detection and Repair: Literature Review and Roadmap" [2024-04] [paper]

• "Pros and Cons! Evaluating ChatGPT on Software Vulnerability" [2024-04] [paper]

• "VulEval: Towards Repository-Level Evaluation of Software Vulnerability Detection" [2024-04] [paper]

• "DLAP: A Deep Learning Augmented Large Language Model Prompting Framework for Software Vulnerability Detection" [2024-05] [paper]

• "Bridging the Gap: A Study of AI-based Vulnerability Management between Industry and Academia" [2024-05] [paper]

• "Bridge and Hint: Extending Pre-trained Language Models for Long-Range Code" [2024-05] [paper]

• "Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study" [2024-05] [paper]

• "LLM-Assisted Static Analysis for Detecting Security Vulnerabilities" [2024-05] [paper]

• "Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning" [2024-06] [ACL 2024 Findings] [paper]

• "Security Vulnerability Detection with Multitask Self-Instructed Fine-Tuning of Large Language Models" [2024-06] [paper]

• "M2CVD: Multi-Model Collaboration for Code Vulnerability Detection" [2024-06] [paper]

• "Towards Effectively Detecting and Explaining Vulnerabilities Using Large Language Models" [2024-06] [paper]

• "Vul-RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG" [2024-06] [paper]

• "Bug In the Code Stack: Can LLMs Find Bugs in Large Python Code Stacks" [2024-06] [paper]

• "Supporting Cross-language Cross-project Bug Localization Using Pre-trained Language Models" [2024-07] [paper]

• "ALPINE: An adaptive language-agnostic pruning method for language models for code" [2024-07] [paper]

• "SCoPE: Evaluating LLMs for Software Vulnerability Detection" [2024-07] [paper]

• "Comparison of Static Application Security Testing Tools and Large Language Models for Repo-level Vulnerability Detection" [2024-07] [paper]

• "Code Structure-Aware through Line-level Semantic Learning for Code Vulnerability Detection" [2024-07] [paper]

• "A Study of Using Multimodal LLMs for Non-Crash Functional Bug Detection in Android Apps" [2024-07] [paper]

• "EaTVul: ChatGPT-based Evasion Attack Against Software Vulnerability Detection" [2024-07] [paper]

• "Evaluating Large Language Models in Detecting Test Smells" [2024-07] [paper]

• "Automated Software Vulnerability Static Code Analysis Using Generative Pre-Trained Transformer Models" [2024-07] [paper]

• "A Qualitative Study on Using ChatGPT for Software Security: Perception vs. Practicality" [2024-08] [paper]

• "Large Language Models for Secure Code Assessment: A Multi-Language Empirical Study" [2024-08] [paper]

• "VulCatch: Enhancing Binary Vulnerability Detection through CodeT5 Decompilation and KAN Advanced Feature Extraction" [2024-08] [paper]

• "Impact of Large Language Models of Code on Fault Localization" [2024-08] [paper]

• "Better Debugging: Combining Static Analysis and LLMs for Explainable Crashing Fault Localization" [2024-08] [paper]

• "Beyond ChatGPT: Enhancing Software Quality Assurance Tasks with Diverse LLMs and Validation Techniques" [2024-09] [paper]

• "CLNX: Bridging Code and Natural Language for C/C++ Vulnerability-Contributing Commits Identification" [2024-09] [paper]

• "Code Vulnerability Detection: A Comparative Analysis of Emerging Large Language Models" [2024-09] [paper]

• "Program Slicing in the Era of Large Language Models" [2024-09] [paper]

• "Generating API Parameter Security Rules with LLM for API Misuse Detection" [2024-09] [paper]

• "Enhancing Fault Localization Through Ordered Code Analysis with LLM Agents and Self-Reflection" [2024-09] [paper]

• "Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Code" [2024-09] [paper]

• "Enhancing Pre-Trained Language Models for Vulnerability Detection via Semantic-Preserving Data Augmentation" [2024-10] [paper]

• "StagedVulBERT: Multi-Granular Vulnerability Detection with a Novel Pre-trained Code Model" [2024-10] [paper]

• "Understanding the AI-powered Binary Code Similarity Detection" [2024-10] [paper]

• "RealVul: Can We Detect Vulnerabilities in Web Applications with LLM?" [2024-10] [paper]

• "Just-In-Time Software Defect Prediction via Bi-modal Change Representation Learning" [2024-10] [paper]

• "DFEPT: Data Flow Embedding for Enhancing Pre-Trained Model Based Vulnerability Detection" [2024-10] [paper]

• "Utilizing Precise and Complete Code Context to Guide LLM in Automatic False Positive Mitigation" [2024-11] [paper]

• "Smart-LLaMA: Two-Stage Post-Training of Large Language Models for Smart Contract Vulnerability Detection and Explanation" [2024-11] [paper]

• "FlexFL: Flexible and Effective Fault Localization with Open-Source Large Language Models" [2024-11] [paper]

• "Breaking the Cycle of Recurring Failures: Applying Generative AI to Root Cause Analysis in Legacy Banking Systems" [2024-11] [paper]

• "Are Large Language Models Memorizing Bug Benchmarks?" [2024-11] [paper]

• "An Empirical Study of Vulnerability Detection using Federated Learning" [2024-11] [paper]

• "Fault Localization from the Semantic Code Search Perspective" [2024-11] [paper]

Malicious Code Detection

• "Deep Android Malware Detection", 2017-03, CODASPY 2017, [paper]

• "A Multimodal Deep Learning Method for Android Malware Detection Using Various Features", 2018-08, IEEE Trans. Inf. Forensics Secur. 2019, [paper]

• "Portable, Data-Driven Malware Detection using Language Processing and Machine Learning Techniques on Behavioral Analysis Reports", 2018-12, Digit. Investig. 2019, [paper]

• "I-MAD: Interpretable Malware Detector Using Galaxy Transformer", 2019-09, Comput. Secur. 2021, [paper]

• "Droidetec: Android Malware Detection and Malicious Code Localization through Deep Learning", 2020-02, [paper]

• "Malicious Code Detection: Run Trace Output Analysis by LSTM", 2021-01, IEEE Access 2021, [paper]

• "Intelligent malware detection based on graph convolutional network", 2021-08, J. Supercomput. 2021, [paper]

• "Malbert: A novel pre-training method for malware detection", 2021-09, Comput. Secur. 2021, [paper]

• "Single-Shot Black-Box Adversarial Attacks Against Malware Detectors: A Causal Language Model Approach", 2021-12, ISI 2021, [paper]

• "M2VMapper: Malware-to-Vulnerability mapping for Android using text processing", 2021-12, Expert Syst. Appl. 2022, [paper]

• "Malware Detection and Prevention using Artificial Intelligence Techniques", 2021-12, IEEE BigData 2021, [paper]

• "An Ensemble of Pre-trained Transformer Models For Imbalanced Multiclass Malware Classification", 2021-12, Comput. Secur. 2022, [paper]

• "EfficientNet convolutional neural networks-based Android malware detection", 2022-01, Comput. Secur. 2022, [paper]

• "Static Malware Detection Using Stacked BiLSTM and GPT-2", 2022-05, IEEE Access 2022, [paper]

• "APT Malicious Sample Organization Traceability Based on Text Transformer Model", 2022-07, PRML 2022, [paper]

• "Self-Supervised Vision Transformers for Malware Detection", 2022-08, IEEE Access 2022, [paper]

• "A Survey of Recent Advances in Deep Learning Models for Detecting Malware in Desktop and Mobile Platforms", 2022-09, ACM Computing Surveys, [paper]

• "Malicious Source Code Detection Using Transformer", 2022-09, [paper]

• "Flexible Android Malware Detection Model based on Generative Adversarial Networks with Code Tensor", 2022-10, CyberC 2022, [paper]

• "MalBERTv2: Code Aware BERT-Based Model for Malware Identification" [2023-03] [Big Data Cogn. Comput. 2023] [paper]

• "GPThreats-3: Is Automatic Malware Generation a Threat?" [2023-05] [SPW 2023] [paper]

• "GitHub Copilot: A Threat to High School Security? Exploring GitHub Copilot's Proficiency in Generating Malware from Simple User Prompts" [2023-08] [ETNCC 2023] [paper]

• "An Attacker's Dream? Exploring the Capabilities of ChatGPT for Developing Malware" [2023-08] [CSET 2023] [paper]

• "Malicious code detection in android: the role of sequence characteristics and disassembling methods" [2023-12] [Int. J. Inf. วินาที. 2023] [paper]

• "Prompt Engineering-assisted Malware Dynamic Analysis Using GPT-4" [2023-12] [paper]

• "Shifting the Lens: Detecting Malware in npm Ecosystem with Large Language Models" [2024-03] [paper]

• "AppPoet: Large Language Model based Android malware detection via multi-view prompt engineering" [2024-04] [paper]

• "Tactics, Techniques, and Procedures (TTPs) in Interpreted Malware: A Zero-Shot Generation with Large Language Models" [2024-07] [paper]

• "DetectBERT: Towards Full App-Level Representation Learning to Detect Android Malware" [2024-08] [paper]

• "PackageIntel: Leveraging Large Language Models for Automated Intelligence Extraction in Package Ecosystems" [2024-09] [paper]

Compiler Optimization

• "Learning Performance-Improving Code Edits" [2023-06] [ICLR 2024 Spotlight] [paper]

• "Large Language Models for Compiler Optimization" [2023-09] [paper]

• "Refining Decompiled C Code with Large Language Models" [2023-10] [paper]

• "Priority Sampling of Large Language Models for Compilers" [2024-02] [paper]

• "Should AI Optimize Your Code? A Comparative Study of Current Large Language Models Versus Classical Optimizing Compilers" [2024-06] [paper]

• "Iterative or Innovative? A Problem-Oriented Perspective for Code Optimization" [2024-06] [paper]

• "Meta Large Language Model Compiler: Foundation Models of Compiler Optimization" [2024-06] [paper]

• "ViC: Virtual Compiler Is All You Need For Assembly Code Search" [2024-08] [paper]

• "Search-Based LLMs for Code Optimization" [2024-08] [paper]

• "E-code: Mastering Efficient Code Generation through Pretrained Models and Expert Encoder Group" [2024-08] [paper]

• "Large Language Models for Energy-Efficient Code: Emerging Results and Future Directions" [2024-10] [paper]

Binary Analysis and Decompilation

• "Using recurrent neural networks for decompilation" [2018-03] [SANER 2018] [paper]

• "Evolving Exact Decompilation" [2018] [paper]

• "Towards Neural Decompilation" [2019-05] [paper]

• "Coda: An End-to-End Neural Program Decompiler" [2019-06] [NeurIPS 2019] [paper]

• "N-Bref : A High-fidelity Decompiler Exploiting Programming Structures" [2020-09] [paper]

• "Neutron: an attention-based neural decompiler" [2021-03] [Cybersecurity 2021] [paper]

• "Beyond the C: Retargetable Decompilation using Neural Machine Translation" [2022-12] [paper]

• "Boosting Neural Networks to Decompile Optimized Binaries" [2023-01] [ACSAC 2022] [paper]

• "SLaDe: A Portable Small Language Model Decompiler for Optimized Assembly" [2023-05] [paper]

• "Nova+: Generative Language Models for Binaries" [2023-11] [paper]

• "CodeArt: Better Code Models by Attention Regularization When Symbols Are Lacking" [2024-11] [paper]

• "LLM4Decompile: Decompiling Binary Code with Large Language Models" [2024-03] [paper]

• "WaDec: Decompile WebAssembly Using Large Language Model" [2024-06] [paper]

• "MAD: Move AI Decompiler to Improve Transparency and Auditability on Non-Open-Source Blockchain Smart Contract" [2024-10] [paper]

• "Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases" [2024-11] [paper]

Commit Message Generation

• "Automated Commit Message Generation with Large Language Models: An Empirical Study and Beyond" [2024-04] [paper]

• "Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings" [2024-10] [paper]

Code Review

• "Using Pre-Trained Models to Boost Code Review Automation" [2022-01] [ICSE 2022] [paper]

• "AUGER: Automatically Generating Review Comments with Pre-training Models" [2022-08] [ESEC/FSE 2022] [paper]

• "Automatic Code Review by Learning the Structure Information of Code Graph" [2023-02] [Sensors] [paper]

• "LLaMA-Reviewer: Advancing Code Review Automation with Large Language Models through Parameter-Efficient Fine-Tuning" [2023-08] [ISSRE 2023] [paper]

• "AI-powered Code Review with LLMs: Early Results" [2024-04] [paper]

• "AI-Assisted Assessment of Coding Practices in Modern Code Review" [2024-05] [paper]

• "A GPT-based Code Review System for Programming Language Learning" [2024-07] [paper]

• "LLM Critics Help Catch LLM Bugs" [2024-06] [paper]

• "Exploring the Capabilities of LLMs for Code Change Related Tasks" [2024-07] [paper]

• "Evaluating Language Models for Generating and Judging Programming Feedback" [2024-07] [paper]

• "Can LLMs Replace Manual Annotation of Software Engineering Artifacts?" [2024-08] [paper]

• "Leveraging Reviewer Experience in Code Review Comment Generation" [2024-09] [paper]

• "CRScore: Grounding Automated Evaluation of Code Review Comments in Code Claims and Smells" [2024-09] [paper]

• "Enhancing Code Annotation Reliability: Generative AI's Role in Comment Quality Assessment Models" [2024-10] [paper]

• "Knowledge-Guided Prompt Learning for Request Quality Assurance in Public Code Review" [2024-10] [paper]

• "Impact of LLM-based Review Comment Generation in Practice: A Mixed Open-/Closed-source User Study" [2024-11] [paper]

• "Prompting and Fine-tuning Large Language Models for Automated Code Review Comment Generation" [2024-11] [paper]

• "Deep Learning-based Code Reviews: A Paradigm Shift or a Double-Edged Sword?" [2024-11] [paper]

• "Redefining Crowdsourced Test Report Prioritization: An Innovative Approach with Large Language Model" [2024-11] [paper]

การวิเคราะห์บันทึก

• "LogStamp: Automatic Online Log Parsing Based on Sequence Labelling" [2022-08] [paper]

• "Log Parsing with Prompt-based Few-shot Learning" [2023-02] [ICSE 2023] [paper]

• "Log Parsing: How Far Can ChatGPT Go?" [2023-06] [ASE 2023] [paper]

• "LogPrompt: Prompt Engineering Towards Zero-Shot and Interpretable Log Analysis" [2023-08] [paper]

• "LogGPT: Exploring ChatGPT for Log-Based Anomaly Detection" [2023-09] [paper]

• "An Assessment of ChatGPT on Log Data" [2023-09] [paper]

• "LILAC: Log Parsing using LLMs with Adaptive Parsing Cache" [2023-10] [paper]

• "LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing" [2024-04] [paper]

• "On the Influence of Data Resampling for Deep Learning-Based Log Anomaly Detection: Insights and Recommendations" [2024-05] [paper]

• "Log Parsing with Self-Generated In-Context Learning and Self-Correction" [2024-06] [paper]

• "Stronger, Faster, and Cheaper Log Parsing with LLMs" [2024-06] [paper]

• "ULog: Unsupervised Log Parsing with Large Language Models through Log Contrastive Units" [2024-06] [paper]

• "Anomaly Detection on Unstable Logs with GPT Models" [2024-06] [paper]

• "LogParser-LLM: Advancing Efficient Log Parsing with Large Language Models" [2024-08] [KDD 2024] [paper]

• "LUK: Empowering Log Understanding with Expert Knowledge from Large Language Models" [2024-09] [paper]

• "A Comparative Study on Large Language Models for Log Parsing" [2024-09] [paper]

• "What Information Contributes to Log-based Anomaly Detection? Insights from a Configurable Transformer-Based Approach" [2024-10] [paper]

• "LogLM: From Task-based to Instruction-based Automated Log Analysis" [2024-10] [paper]

Software Configuration

• "Configuration Validation with Large Language Models" [2023-10] [paper]

• "CloudEval-YAML: A Practical Benchmark for Cloud Configuration Generation" [2023-11] [paper]

• "Can LLMs Configure Software Tools" [2023-12] [paper]

• "LuaTaint: A Static Analysis System for Web Configuration Interface Vulnerability of Internet of Things Devices" [2024-02] [IOT] [paper]

• "LLM-Based Misconfiguration Detection for AWS Serverless Computing" [2024-11] [paper]

• "LogLLM: Log-based Anomaly Detection Using Large Language Models" [2024-11] [paper]

Software Modeling

• "Towards using Few-Shot Prompt Learning for Automating Model Completion" [2022-12] [paper]

• "Model Generation from Requirements with LLMs: an Exploratory Study" [2024-04] [paper]

• "How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts" [2024-04] [paper]

• "Leveraging Large Language Models for Software Model Completion: Results from Industrial and Public Datasets" [2024-06] [paper]

• "Studying and Benchmarking Large Language Models For Log Level Suggestion" [2024-10] [paper]

• "A Model Is Not Built By A Single Prompt: LLM-Based Domain Modeling With Question Decomposition" [2024-10] [paper]

• "On the Utility of Domain Modeling Assistance with Large Language Models" [2024-10] [paper]

• "On the use of Large Language Models in Model-Driven Engineering" [2024-10] [paper]

• "LLM as a code generator in Agile Model Driven Development" [2024-10] [paper]

Requirement Engineering

• "A Transformer-based Approach for Abstractive Summarization of Requirements from Obligations in Software Engineering Contracts" [2023-09] [RE 2023] [paper]

• "Advancing Requirements Engineering through Generative AI: Assessing the Role of LLMs" [2023-10] [paper]

• "Requirements Engineering using Generative AI: Prompts and Prompting Patterns" [2023-11] [paper]

• "Prioritizing Software Requirements Using Large Language Models" [2024-04] [paper]

• "Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks" [2024-04] [paper]

• "Enhancing Legal Compliance and Regulation Analysis with Large Language Models" [2024-04] [paper]

• "MARE: Multi-Agents Collaboration Framework for Requirements Engineering" [2024-05] [paper]

• "Natural Language Processing for Requirements Traceability" [2024-05] [paper]

• "Multilingual Crowd-Based Requirements Engineering Using Large Language Models" [2024-08] [paper]

• "From Specifications to Prompts: On the Future of Generative LLMs in Requirements Engineering" [2024-08] [paper]

• "Leveraging LLMs for the Quality Assurance of Software Requirements" [2024-08] [paper]

• "Generative AI for Requirements Engineering: A Systematic Literature Review" [2024-09] [paper]

• "A Fine-grained Sentiment Analysis of App Reviews using Large Language Models: An Evaluation Study" [2024-09] [paper]

• "Leveraging Large Language Models for Predicting Cost and Duration in Software Engineering Projects" [2024-09] [paper]

• "Privacy Policy Analysis through Prompt Engineering for LLMs" [2024-09] [paper]

• "Exploring Requirements Elicitation from App Store User Reviews Using Large Language Models" [2024-09] [paper]

• "LLM-Cure: LLM-based Competitor User Review Analysis for Feature Enhancement" [2024-09] [paper]

• "Automatic Instantiation of Assurance Cases from Patterns Using Large Language Models" [2024-10] [paper]

• "Whose fault is it anyway? SILC: Safe Integration of LLM-Generated Code" [2024-10] [paper]

• "Assured Automatic Programming via Large Language Models" [2024-10] [paper]

• "Does GenAI Make Usability Testing Obsolete?" [2024-11] [paper]

• "Exploring LLMs for Verifying Technical System Specifications Against Requirements" [2024-11] [paper]

• "Towards the LLM-Based Generation of Formal Specifications from Natural-Language Contracts: Early Experiments with Symboleo" [2024-11] [paper]

6. Analysis of AI-Generated Code

Security and Vulnerabilities

• "You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion" [2021-08] [USENIX Security Symposium 2021] [paper]

• "Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?" [2022-04] [Empir. Softw. Eng.] [paper]

• "Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants" [2022-08] [USENIX Security Symposium 2023] [paper]

• "Do Users Write More Insecure Code with AI Assistants?" [2022-1] [CCS 2023] [paper]

• "Large Language Models for Code: Security Hardening and Adversarial Testing" [2023-02] [CCS 2023] [paper]

• "Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models" [2023-12] [paper]

• "CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion" [2024-03] [ACL 2024 Findings] [paper]

• "Just another copy and paste? Comparing the security vulnerabilities of ChatGPT generated code and StackOverflow answers" [2024-03] [paper]

• "DeVAIC: A Tool for Security Assessment of AI-generated Code" [2024-04] [paper]

• "CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models" [2024-04] [paper]

• "LLMs in Web-Development: Evaluating LLM-Generated PHP code unveiling vulnerabilities and limitations" [2024-04] [paper]

• "Do Neutral Prompts Produce Insecure Code? FormAI-v2 Dataset: Labelling Vulnerabilities in Code Generated by Large Language Models" [2024-04] [paper]

• "Codexity: Secure AI-assisted Code Generation" [2024-05] [paper]

• "Measuring Impacts of Poisoning on Model Parameters and Embeddings for Large Language Models of Code" [2024-05] [paper]

• "An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection" [2024-06] [paper]

• "Is Your AI-Generated Code Really Secure? Evaluating Large Language Models on Secure Code Generation with CodeSecEval" [2024-07] [paper]

• "Prompting Techniques for Secure Code Generation: A Systematic Investigation" [2024-07] [paper]

• "TAPI: Towards Target-Specific and Adversarial Prompt Injection against Code LLMs" [2024-07] [paper]

• "MaPPing Your Model: Assessing the Impact of Adversarial Attacks on LLM-based Programming Assistants" [2024-07] [paper]

• "Eliminating Backdoors in Neural Code Models via Trigger Inversion" [2024-08] [paper]

• ""You still have to study" -- On the Security of LLM generated code" [2024-08] [paper]

• "How Well Do Large Language Models Serve as End-to-End Secure Code Producers?" [2024-08] [paper]

• "While GitHub Copilot Excels at Coding, Does It Ensure Responsible Output?" [2024-08] [paper]

• "PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)" [2024-09] [paper]

• "RMCBench: Benchmarking Large Language Models' Resistance to Malicious Code" [2024-09] [paper]

• "Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation" [2024-09] [paper]

• "Demonstration Attack against In-Context Learning for Code Intelligence" [2024-10] [paper]

• "Hallucinating AI Hijacking Attack: Large Language Models and Malicious Code Recommenders" [2024-10] [paper]

• "SecCodePLT: A Unified Platform for Evaluating the Security of Code GenAI" [2024-10] [paper]

• "Security of Language Models for Code: A Systematic Literature Review" [2024-10] [paper]

• "RedCode: Risky Code Execution and Generation Benchmark for Code Agents" [2024-11] [paper]

• "ProSec: Fortifying Code LLMs with Proactive Security Alignment" [2024-11] [paper]

Correctness

• "An Empirical Evaluation of GitHub Copilot's Code Suggestions" [2022-05] [MSR 2022] [paper]

• "Large Language Models and Simple, Stupid Bugs" [2023-03] [MSR 2023] [paper]

• "Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT" [2023-04] [paper]

• "No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPT" [2023-08] [paper]

• "The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?" [2024-02] [ACL 2024 Findings] [paper]

• "Bugs in Large Language Models Generated Code: An Empirical Study" [2024-03] [paper]

• "ChatGPT Incorrectness Detection in Software Reviews" [2024-03] [paper]

• "Validating LLM-Generated Programs with Metamorphic Prompt Testing" [2024-06] [paper]

• "Where Do Large Language Models Fail When Generating Code?" [2024-06] [paper]

• "GitHub Copilot: the perfect Code compLeeter?" [2024-06] [paper]

• "What's Wrong with Your Code Generated by Large Language Models? An Extensive Study" [2024-07] [paper]

• "Uncovering Weaknesses in Neural Code Generation" [2024-07] [paper]

• "Understanding Defects in Generated Codes by Language Models" [2024-08] [paper]

• "CodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation" [2024-08] [paper]

• "Examination of Code generated by Large Language Models" [2024-08] [paper]

• "Fixing Code Generation Errors for Large Language Models" [2024-09] [paper]

• "Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generation" [2024-09] [paper]

• "Insights from Benchmarking Frontier Language Models on Web App Code Generation" [2024-09] [paper]

• "Evaluating the Performance of Large Language Models in Competitive Programming: A Multi-Year, Multi-Grade Analysis" [2024-09] [paper]

• "A Case Study of Web App Coding with OpenAI Reasoning Models" [2024-09] [paper]

• "CodeJudge: Evaluating Code Generation with Large Language Models" [2024-10] [paper]

• "An evaluation of LLM code generation capabilities through graded exercises" [2024-10] [paper]

• "A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?" [2024-11] [paper]

• "Evaluating ChatGPT-3.5 Efficiency in Solving Coding Problems of Different Complexity Levels: An Empirical Analysis" [2024-11] [paper]

• "LLM4DS: Evaluating Large Language Models for Data Science Code Generation" [2024-11] [paper]

• "A Preliminary Study of Multilingual Code Language Models for Code Generation Task Using Translated Benchmarks" [2024-11] [paper]

Hallucination

• "Exploring and Evaluating Hallucinations in LLM-Powered Code Generation" [2024-04] [paper]

• "CodeHalu: Code Hallucinations in LLMs Driven by Execution-based Verification" [2024-04] [paper]

• "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs" [2024-06] [paper]

• "Code Hallucination" [2024-07] [paper]

• "On Mitigating Code LLM Hallucinations with API Documentation" [2024-07] [paper]

• "CodeMirage: Hallucinations in Code Generated by Large Language Models" [2024-08] [paper]

• "LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation" [2024-09] [paper]

• "Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Code" [2024-10] [paper]

• "ETF: An Entity Tracing Framework for Hallucination Detection in Code Summaries" [2024-10] [paper]

ประสิทธิภาพ

• "On Evaluating the Efficiency of Source Code Generated by LLMs" [2024-04] [paper]

• "A Controlled Experiment on the Energy Efficiency of the Source Code Generated by Code Llama" [2024-05] [paper]

• "From Effectiveness to Efficiency: Comparative Evaluation of Code Generated by LCGMs for Bilingual Programming Questions" [2024-06] [paper]

• "How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark" [2024-06] [paper]

• "ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?" [2024-07] [paper]

• "A Performance Study of LLM-Generated Code on Leetcode" [2024-07] [paper]

• "Evaluating Language Models for Efficient Code Generation" [2024-08] [paper]

• "Effi-Code: Unleashing Code Efficiency in Language Models" [2024-10] [paper]

• "Rethinking Code Refinement: Learning to Judge Code Efficiency" [2024-10] [paper]

• "Generating Energy-efficient code with LLMs" [2024-11] [paper]

• "An exploration of the effect of quantisation on energy consumption and inference time of StarCoder2" [2024-11] [paper]

Robustness

• "Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain" [2023-10] [paper]

• "Do Large Code Models Understand Programming Concepts? A Black-box Approach" [2024-02] [ICML 2024] [paper]

• "Syntactic Robustness for LLM-based Code Generation" [2024-04] [paper]

• "NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variations" [2024-06] [paper]

• "An Empirical Study on Capability of Large Language Models in Understanding Code Semantics" [2024-07] [paper]

• "Comparing Robustness Against Adversarial Attacks in Code Generation: LLM-Generated vs. Human-Written" [2024-11] [paper]

การตีความได้

• "A Critical Study of What Code-LLMs (Do Not) Learn" [2024-06] [ACL 2024 Findings] [paper]

• "Looking into Black Box Code Language Models" [2024-07] [paper]

• "DeepCodeProbe: Towards Understanding What Models Trained on Code Learn" [2024-07] [paper]

• "Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations" [2024-07] [paper]

API Usage

• "How and Why LLMs Use Deprecated APIs in Code Completion? An Empirical Study" [2024-06] [paper]

• "Is ChatGPT a Good Software Librarian? An Exploratory Study on the Use of ChatGPT for Software Library Recommendations" [2024-08] [paper]

• "A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How" [2024-09] [paper]

• "AutoAPIEval: A Framework for Automated Evaluation of LLMs in API-Oriented Code Generation" [2024-09] [paper]

ความเป็นส่วนตัว

• "Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach" [2024-04] [paper]

• "CodeCipher: Learning to Obfuscate Source Code Against LLMs" [2024-10] [paper]

• "Decoding Secret Memorization in Code LLMs Through Token-Level Characterization" [2024-10] [paper]

อคติ

• "Exploring Multi-Lingual Bias of Large Code Models in Code Generation" [2024-04] [paper]

• "Mitigating Gender Bias in Code Large Language Models via Model Editing" [2024-10] [paper]

• "Bias Unveiled: Investigating Social Bias in LLM-Generated Code" [2024-11] [paper]

AI-Generated Code Detection

• "Zero-Shot Detection of Machine-Generated Codes" [2023-10] [paper]

• "CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code" [2024-04] [paper]

• "ChatGPT Code Detection: Techniques for Uncovering the Source of Code" [2024-05] [paper]

• "Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting" [2024-05] [paper]

• "Automatic Detection of LLM-generated Code: A Case Study of Claude 3 Haiku" [2024-09] [paper]

• "An Empirical Study on Automatically Detecting AI-Generated Source Code: How Far Are We?" [2024-11] [paper]

• "Distinguishing LLM-generated from Human-written Code by Contrastive Learning" [2024-11] [paper]

คนอื่น

• "Who Wrote this Code? Watermarking for Code Generation" [2023-05] [ACL 2024] [paper]

• "Testing the Effect of Code Documentation on Large Language Model Code Understanding" [2024-04] [paper]

• "Automated Creation of Source Code Variants of a Cryptographic Hash Function Implementation Using Generative Pre-Trained Transformer Models" [2024-04] [paper]

• "Evaluation of the Programming Skills of Large Language Models" [2024-05] [paper]

• "Where Are Large Language Models for Code Generation on GitHub?" [2024-06] [paper]

• "Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Models" [2024-06] [paper]

• "Benchmarking Language Model Creativity: A Case Study on Code Generation" [2024-07] [paper]

• "Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models" [2024-07] [paper]

• "Is Functional Correctness Enough to Evaluate Code Language Models? Exploring Diversity of Generated Codes" [2024-08] [paper]

• "Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programming" [2024-08] [paper]

• "A Survey on Evaluating Large Language Models in Code Generation Tasks" [2024-08] [paper]

• "An exploratory analysis of Community-based Question-Answering Platforms and GPT-3-driven Generative AI: Is it the end of online community-based learning?" [2024-09] [paper]

• "Code Generation and Algorithmic Problem Solving Using Llama 3.1 405B" [2024-09] [paper]

• "Benchmarking ChatGPT, Codeium, and GitHub Copilot: A Comparative Study of AI-Driven Programming and Debugging Assistants" [2024-09] [paper]

• "Model Editing for LLMs4Code: How Far are We?" [2024-11] [paper]

• "An Empirical Study on LLM-based Agents for Automated Bug Fixing" [2024-11] [paper]

• "Precision or Peril: Evaluating Code Quality from Quantized Large Language Models" [2024-11] [paper]

7. Human-LLM Interaction

• "Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models" [2022-04] [CHI EA 2022] [paper]

• "Grounded Copilot: How Programmers Interact with Code-Generating Models" [2022-06] [OOPSLA 2023] [paper]

• "Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming" [2022-10] [paper]

• "The Impact of AI on Developer Productivity: Evidence from GitHub Copilot" [2023-02] [paper]

• "The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development" [2023-02] [IUI 2023] [paper]

• ""It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers" [2023-04] [ACM TCHI] [paper]

• "DevGPT: Studying Developer-ChatGPT Conversations" [2023-08] [paper]

• "How Do Analysts Understand and Verify AI-Assisted Data Analyses?" [2023-09] [paper]

• "How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment" [2023-09] [Koli Calling 2023] [paper]

• "Conversational Challenges in AI-Powered Data Science: Obstacles, Needs, and Design Opportunities" [2023-10] [paper]

• "The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers" [2024-04] [paper]

• "Unlocking Adaptive User Experience with Generative AI" [2024-04] [paper]

• "BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebooks" [2024-04] [paper]

• "How far are AI-powered programming assistants from meeting developers' needs?" [2024-04] [paper]

• "Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice" [2024-04] [paper]

• "The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement but Increased Adopters Exam Performances" [2024-04] [paper]

• "amplified.dev: a living document that begins to sketch a vision for a future where developers are amplified, not automated" [2024-05] [paper]

• "Sketch Then Generate: Providing Incremental User Feedback and Guiding LLM Code Generation through Language-Oriented Code Sketches" [2024-05] [paper]

• "Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns" [2024-05] [paper]

• "Full Line Code Completion: Bringing AI to Desktop" [2024-05] [paper]

• "Developers' Perceptions on the Impact of ChatGPT in Software Development: A Survey" [2024-05] [paper]

• "A Transformer-Based Approach for Smart Invocation of Automatic Code Completion" [2024-05] [paper]

• "A Study on Developer Behaviors for Validating and Repairing LLM-Generated Code Using Eye Tracking and IDE Actions" [2024-05] [paper]

• "Analyzing Chat Protocols of Novice Programmers Solving Introductory Programming Tasks with ChatGPT" [2024-05] [paper]

• "Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent" [2024-05] [paper]

• "Learning Task Decomposition to Assist Humans in Competitive Programming" [2024-06] [ACL 2024] [paper]

• "Impact of AI-tooling on the Engineering Workspace" [2024-06] [paper]

• "Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forward" [2024-06] [paper]

• "Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debugging" [2024-06] [paper]

• "Transforming Software Development: Evaluating the Efficiency and Challenges of GitHub Copilot in Real-World Projects" [2024-06] [paper]

• "Let the Code LLM Edit Itself When You Edit the Code" [2024-07] [paper]

• "Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation" [2024-07] [paper]

• "How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course" [2024-07] [paper]

• "Can Developers Prompt? A Controlled Experiment for Code Documentation Generation" [2024-08] [paper]

• "The Impact of Generative AI-Powered Code Generation Tools on Software Engineer Hiring: Recruiters' Experiences, Perceptions, and Strategies" [2024-09] [paper]

• "Investigating the Role of Cultural Values in Adopting Large Language Models for Software Engineering" [2024-09] [paper]

• "The Impact of Large Language Models on Open-source Innovation: Evidence from GitHub Copilot" [2024-09] [paper]

• ""I Don't Use AI for Everything": Exploring Utility, Attitude, and Responsibility of AI-empowered Tools in Software Development" [2024-09] [paper]

• "Harnessing the Potential of Gen-AI Coding Assistants in Public Sector Software Development" [2024-09] [paper]

• "Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Tasks" [2024-10] [paper]

• "Code-Survey: An LLM-Driven Methodology for Analyzing Large-Scale Codebases" [2024-10] [paper]

• "The potential of LLM-generated reports in DevSecOps" [2024-10] [paper]

• "The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot" [2024-10] [paper]

• "Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learning" [2024-10] [paper]

• "One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks" [2024-10] [paper]

• "UniAutoML: A Human-Centered Framework for Unified Discriminative and Generative AutoML with Large Language Models" [2024-10] [paper]

• "How much does AI impact development speed? An enterprise-based randomized controlled trial" [2024-10] [paper]

• "Understanding the Effect of Algorithm Transparency of Model Explanations in Text-to-SQL Semantic Parsing" [2024-10] [paper]

• "Dear Diary: A randomized controlled trial of Generative AI coding tools in the workplace" [2024-10] [paper]

• "LLMs are Imperfect, Then What? An Empirical Study on LLM Failures in Software Engineering" [2024-11] [paper]

• "Human-In-the-Loop Software Development Agents" [2024-11] [paper]

8. Datasets

8.1 Pretraining

1. CodeSearchNet : "CodeSearchNet Challenge: Evaluating the State of Semantic Code Search" [2019-09] [paper] [repo] [data]

2. The Pile : "The Pile: An 800GB Dataset of Diverse Text for Language Modeling" [2020-12], [paper] [data]

3. CodeParrot , 2022-02, [data]

4. The Stack : "The Stack: 3 TB of permissively licensed source code" [2022-11] [paper] [data]

5. ROOTS : "The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset" [2023-03] [NeurIPS 2022 Datasets and Benchmarks Track] [paper] [data]

6. The Stack v2 : "StarCoder 2 and The Stack v2: The Next Generation" [2024-02] [paper] [data]

8.2 Benchmarks

Integrated Benchmarks

• CodeXGLUE : "CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation" [2021-02] [NeurIPS Datasets and Benchmarks 2021] [paper] [repo] [data]

• CodefuseEval : "CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model" [2023-10] [paper] [repo]

• CodeScope : "CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation" [2023-11] [ACL 2024] [paper] [repo]

• CodeEditorBench : "CodeEditorBench: Evaluating Code Editing Capability of Large Language Models" [2024-04] [paper] [repo]

• Long Code Arena : "Long Code Arena: a Set of Benchmarks for Long-Context Code Models" [2024-06] [paper] [repo]

• CodeRAG-Bench : "CodeRAG-Bench: Can Retrieval Augment Code Generation?" [2024-06] [paper] [repo]

• LiveBench : "LiveBench: A Challenging, Contamination-Free LLM Benchmark" [2024-06] [paper] [repo]

• DebugEval : "Enhancing the Code Debugging Ability of LLMs via Communicative Agent Based Data Refinement" [2024-08] [paper] [repo]

Program Synthesis

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2018-02	LREC 2018	NL2Bash	9305	ทุบตี	"NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating System" [paper] [data]
2018-08	EMNLP 2018	CONCODE	104K	ชวา	"Mapping Language to Code in Programmatic Context" [paper] [data]
2019-10	EMNLP-IJCNLP 2019	น้ำผลไม้	1.5M/3725 *	งูหลาม	"JuICe: A Large Scale Distantly Supervised Dataset for Open Domain Context-based Code Generation" [paper] [data]
2021-05	NeurIPS 2021	APPS	10,000	งูหลาม	"Measuring Coding Challenge Competence With APPS" [paper] [data]
2021-07	arXiv	HumanEval	164	งูหลาม	"Evaluating Large Language Models Trained on Code" [paper] [data]
2021-08	arXiv	MBPP/MathQA-Python	974/23914	งูหลาม	"Program Synthesis with Large Language Models" [paper] [MBPP] [MathQA-Python]
2021-08	ACL/IJCNLP 2021	PlotCoder	40797	งูหลาม	"PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context" [paper] [data]
2022-01	arXiv	DSP	1119	งูหลาม	"Training and Evaluating a Jupyter Notebook Data Science Assistant" [paper] [data]
2022-02	ศาสตร์	CodeContests	13610	C++, Python, Java	"Competition-Level Code Generation with AlphaCode" [paper] [data]
2022-03	EACL 2023 Findings	MCoNaLa	896	งูหลาม	"MCoNaLa: A Benchmark for Code Generation from Multiple Natural Languages" [paper] [data]
2022-06	arXiv	AixBench	336	ชวา	"AixBench: A Code Generation Benchmark Dataset" [paper] [data]
2022-08	IEEE Trans. Software Engineering	หลายรายการ			"MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generation", [paper] [data]
2022-10	ICLR 2023	MBXP	12.4K	Python, Java, JS, TypeScript, Go, C#, PHP, Ruby, Kotlin, C++, Perl, Scala, Swift	"Multi-lingual Evaluation of Code Generation Models" [paper] [data]
2022-10	ICLR 2023	Multilingual HumanEval	1.9K	Python, Java, JS, TypeScript, Go, C#, PHP, Ruby, Kotlin, Perl, Scala, Swift	"Multi-lingual Evaluation of Code Generation Models" [paper] [data]
2022-10	ICLR 2023	MathQA-X	5.6K	Python, Java, JS	"Multi-lingual Evaluation of Code Generation Models" [paper] [data]
2022-11	arXiv	ExeDS	534	งูหลาม	"Execution-based Evaluation for Data Science Code Generation Models" [paper] [data]
2022-11	arXiv	DS-1000	1,000	งูหลาม	"DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation" [paper] [data]
2022-12	arXiv	ODEX	945	งูหลาม	"Execution-Based Evaluation for Open-Domain Code Generation" [paper] [data]
2023-02	arXiv	CoderEval	460	Python, Java	"CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Models" [paper] [data]
2023-03	ACL 2024	xCodeEval	5.5M	C, C#, C++, Go, Java, JS, Kotlin, PHP, Python, Ruby, Rust	"XCodeEval: An Execution-based Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval" [paper] [data]
2023-03	arXiv	HumanEval-X	820	Python, C++, Java, JS, Go	"CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X" [paper] [data]
2023-05	arXiv	HumanEval+	164	งูหลาม	"Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation" [paper] [data]
2023-06	ACL 2024 Findings	StudentEval	1749 $^dagger$	งูหลาม	"StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code" [paper] [data]
2023-08	ICLR 2024 Spotlight	HumanEvalPack	984	Python, JS, Go, Java, C++, Rust	"OctoPack: Instruction Tuning Code Large Language Models" [paper] [data]
2023-06	NeurIPS 2023	DotPrompts	10538 $^ddagger$	ชวา	"Guiding Language Models of Code with Global Context using Monitors" [paper] [data]
2023-09	arXiv	CodeApex	476	C++	"CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models" [paper] [data]
2023-09	arXiv	VerilogEval	8645/156 $^diamond$	Verilog	"VerilogEval: Evaluating Large Language Models for Verilog Code Generation" [paper] [data]
2023-11	arXiv	ML-Bench	10040	ทุบตี	"ML-Bench: Large Language Models Leverage Open-source Libraries for Machine Learning Tasks" [paper] [data]
2023-12	arXiv	TACO	26,433	งูหลาม	"TACO: Topics in Algorithmic COde generation dataset" [paper] [data]
2024-01	HPDC	ParEval	420	C++, CUDA, HIP	"Can Large Language Models Write Parallel Code?" [paper] [data]
2024-02	ACL 2024 Findings	OOP	431	งูหลาม	"OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models" [paper] [data]
2024-02	LREC-COLING 2024	HumanEval-XL	22080	23NL, 12PL	"HumanEval-XL: A Multilingual Code Generation Benchmark for Cross-lingual Natural Language Generalization" [paper] [data]
2024-04	arXiv	USACO	307	งูหลาม	"Can Language Models Solve Olympiad Programming?" [paper] [data]
2024-04	LREC-COLING 2024	PECC	2396	งูหลาม	"PECC: Problem Extraction and Coding Challenges" [paper] [data]
2024-04	arXiv	CodeGuard+	23	Python, C	"Constrained Decoding for Secure Code Generation" [paper] [data]
2024-05	ACL 2024 Findings	NaturalCodeBench	402	Python, Java	"NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Prompts" [paper] [data]
2024-05	arXiv	MHPP	140	งูหลาม	"MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generation" [paper] [repo]
2024-06	arXiv	VHDL-Eval	202	VHDL	"VHDL-Eval: A Framework for Evaluating Large Language Models in VHDL Code Generation" [paper]
2024-06	arXiv	AICoderEval	492	งูหลาม	"AICoderEval: Improving AI Domain Code Generation of Large Language Models" [paper] [data]
2024-06	arXiv	VersiCode	98,692	งูหลาม	"VersiCode: Towards Version-controllable Code Generation" [paper] [data]
2024-06	IEEE AITest 2024	ScenEval	12,864	ชวา	"ScenEval: A Benchmark for Scenario-Based Evaluation of Code Generation" [paper]
2024-06	arXiv	BigCodeBench	1,140	งูหลาม	"BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions" [paper] [data]
2024-07	arXiv	CodeUpdateArena	670	งูหลาม	"CodeUpdateArena: Benchmarking Knowledge Editing on API Updates" [paper] [data]
2024-07	arXiv	LBPP	161	งูหลาม	"On Leakage of Code Generation Evaluation Datasets" [paper] [data]
2024-07	arXiv	NoviCode	150	งูหลาม	"NoviCode: Generating Programs from Natural Language Utterances by Novices" [paper] [data]
2024-07	arXiv	Case2Code	1.3M	งูหลาม	"Case2Code: Learning Inductive Reasoning with Synthetic Data" [paper] [data]
2024-07	arXiv	SciCode	338	งูหลาม	"SciCode: A Research Coding Benchmark Curated by Scientists" [paper] [data]
2024-07	arXiv	auto-regression	460	งูหลาม	"Generating Unseen Code Tests In Infinitum" [paper]
2024-07	arXiv	WebApp1K	1,000	จาวาสคริปต์	"WebApp1K: A Practical Code-Generation Benchmark for Web App Development" [paper] [data]
2024-08	ACL 2024 Findings	CodeInsight	3409	งูหลาม	"CodeInsight: A Curated Dataset of Practical Coding Solutions from Stack Overflow" [paper] [data]
2024-08	arXiv	DomainEval	2454	งูหลาม	"DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation" [paper] [data]
2024-09	arXiv	ComplexCodeEval	7184/3897	Python/Java	"ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code" [paper] [data]
2024-09	ASE 2024	CoCoNote	58221	Python Notebook	"Contextualized Data-Wrangling Code Generation in Computational Notebooks" [paper] [data]
2024-10	arXiv	unnamed	77	งูหลาม	"Evaluation of Code LLMs on Geospatial Code Generation" [paper] [data]
2024-10	arXiv	mHumanEval	836,400	25PL, 204NL	"mHumanEval -- A Multilingual Benchmark to Evaluate Large Language Models for Code Generation" [paper] [data]
2024-10	arXiv	FeatEng	103	งูหลาม	"Can Models Help Us Create Better Models? Evaluating LLMs as Data Scientists" [paper] [data]
2024-11	arXiv	GitChameleon	116	งูหลาม	"GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Models" [paper] [data]

* Automatically mined/human-annotated

$^dagger$ 1749 prompts for 48 problems

$^ddagger$ 10538 prompts for 1420 problems

$^diamond$ Machine/human prompts

Visually Grounded Program Synthesis

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2024-04	arXiv	MMCode	3548	งูหลาม	"MMCode: Evaluating Multi-Modal Code Large Language Models with Visually Rich Programming Problems" [paper] [data]
2024-05	arXiv	Plot2Code	132	งูหลาม	"Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots" [paper] [data]
2024-06	arXiv	ChartMimic	1,000	งูหลาม	"ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation" [paper] [data]
2024-10	arXiv	HumanEval-V	108	งูหลาม	"HumanEval-V: Evaluating Visual Understanding and Reasoning Abilities of Large Multimodal Models Through Coding Tasks" [paper] [data]
2024-10	arXiv	TurtleBench	260	งูหลาม	"TurtleBench: A Visual Programming Benchmark in Turtle Geometry" [paper] [data]

Code Reasoning and QA

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2021-09	EMNLP 2021 Findings	CodeQA	120K/70K	Java/Python	"CodeQA: A Question Answering Dataset for Source Code Comprehension" [paper] [data]
2022-10	NAACL 2022	CS1QA	9237	งูหลาม	"CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course" [paper] [data]
2023-09	arXiv	CodeApex	250	C++	"CodeApex: A Bilingual Programming Evaluation Benchmark for Large Language Models" [paper] [data]
2024-01	ICML 2024	CRUXEval	800	งูหลาม	"CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution" [paper] [data]
2024-05	arXiv	PythonIO	2650	งูหลาม	"Multiple-Choice Questions are Efficient and Robust LLM Evaluators" [paper] [data]
2024-05	arXiv	StaCCQA	270K	งูหลาม	"Aligning LLMs through Multi-perspective User Preference Ranking-based Feedback for Programming Question Answering" [paper] [data]
2024-06	arXiv	RepoQA	500	Python, C++, Java, Rust, TypeScript	"RepoQA: Evaluating Long Context Code Understanding" [paper] [data]
2024-08	arXiv	CruxEval-X	12.6K	19	"CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution" [paper] [data]
2024-09	arXiv	SpecEval	204	ชวา	"SpecEval: Evaluating Code Comprehension in Large Language Models via Program Specifications" [paper] [data]
2024-10	arXiv	CodeMMLU	19912	13	"CodeMMLU: A Multi-Task Benchmark for Assessing Code Understanding Capabilities of CodeLLMs" [paper] [data]
2024-11	arXiv	unnamed	80232	งูหลาม	"Leveraging Large Language Models in Code Question Answering: Baselines and Issues" [paper] [data]

Text-to-SQL

• "Deep learning driven natural languages text to SQL query conversion: A survey", 2022-08, arXiv, [paper]
• "Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect", 2022-08, COLING 2022, [paper]
• "A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions", 2022-08, arXiv, [paper]
• "A survey on deep learning approaches for text-to-SQL", 2023-01, VLDB J., [paper]

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2017-08	arXiv	WikiSQL	80654		"Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning" [paper] [data]
2018-06	CL 2018	Advising	4570		"Improving Text-to-SQL Evaluation Methodology" [paper] [data]
2018-09	EMNLP 2018	แมงมุม	10181		"Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task" [paper] [data]
2019-06	ACL 2019	SParC	12726		"SParC: Cross-Domain Semantic Parsing in Context" [paper] [data]
2019-07	WWW 2020	MIMICSQL	10,000		"Text-to-SQL Generation for Question Answering on Electronic Medical Records" [paper] [data]
2019-09	EMNLP-IJCNLP 2019	CoSQL	15598		"CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases" [paper] [data]
2020-05	LREC 2020	Criteria-to-SQL	2003		"Dataset and Enhanced Model for Eligibility Criteria-to-SQL Semantic Parsing" [paper] [data]
2020-10	EMNLP 2020 Findings	Squall	11276		"On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries" [paper] [data]
2020-10	NAACL-HLT 2021	Spider-Realistic	508		"Structure-Grounded Pretraining for Text-to-SQL" [paper] [data]
2021-06	ACL/IJCNLP 2021	Spider-Syn	8034		"Towards Robustness of Text-to-SQL Models against Synonym Substitution" [paper] [data]
2021-06	NLP4Prog 2021	SEDE	12023		"Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data" [paper] [data]
2021-06	ACL/IJCNLP 2021	KaggleDBQA	400		"KaggleDBQA: Realistic Evaluation of Text-to-SQL Parsers" [paper] [data]
2021-09	EMNLP	Spider-DK	535		"Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization" [paper] [data]
2022-05	NAACL 2022 Findings	Spider-SS/CG	8034/45599		"Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignment" [paper] [data]
2023-05	arXiv	นก	12751		"Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs" [paper] [data]
2023-06	ACL 2023	XSemPLR	24.4K		"XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations" [paper] [data]
2024-05	ACL 2024 Findings	EHR-SeqSQL	31669		"EHR-SeqSQL : A Sequential Text-to-SQL Dataset For Interactively Exploring Electronic Health Records" [paper]
2024-06	NAACL 2024	BookSQL	100K		"BookSQL: A Large Scale Text-to-SQL Dataset for Accounting Domain" [paper] [data]
2024-08	ACL 2024 Findings	MultiSQL	9257		"MultiSQL: A Schema-Integrated Context-Dependent Text2SQL Dataset with Diverse SQL Operations" [paper] [data]
2024-09	arXiv	BEAVER	93		"BEAVER: An Enterprise Benchmark for Text-to-SQL" [paper]
2024-10	arXiv	PRACTIQ	2812		"PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries" [paper]
2024-10	arXiv	ทวิ	239		"BIS: NL2SQL Service Evaluation Benchmark for Business Intelligence Scenarios" [paper] [data]
2024-11	arXiv	Spider 2.0	632		"Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows" [paper] [data]

Code Translation

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2020-06	NeurIPS 2020	Transcoder GeeksforGeeks	1.4K	C++, Java, Python	"Unsupervised Translation of Programming Languages" [paper] [data]
2021-02	NeurIPS Datasets and Benchmarks 2021	CodeTrans	11.8K	Java, C#	"CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation" [paper] [data]
2021-08	ACL 2023 Findings	อวตาร	9515	Java, Python	"AVATAR: A Parallel Corpus for Java-Python Program Translation" [paper] [data]
2022-06	AAAI 2022	ค่าใช้จ่าย	132K	C++, Java, Python, C#, JS, PHP, C	"Multilingual Code Snippets Training for Program Translation" [paper] [data]
2022-06	arXiv	XLCoST	567K	C++, Java, Python, C#, JS, PHP, C	"XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligence" [paper] [data]
2023-03	arXiv	xCodeEval	5.6M	C, C#, C++, Go, Java, JS, Kotlin, PHP, Python, Ruby, Rust	"xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval" [paper] [data]
2023-03	arXiv	HumanEval-X	1640	Python, C++, Java, JS, Go	"CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Evaluations on HumanEval-X" [paper] [data]
2023-08	arXiv	G-TransEval	4000	C++, Java, C#, JS, Python	"On the Evaluation of Neural Code Translation: Taxonomy and Benchmark" [paper] [data]
2023-10	arXiv	CodeTransOcean	270.5K	45	"CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation" [paper] [data]
2024-11	arXiv	Classeval-T	94	Python, Java, C++	"Escalating LLM-based Code Translation Benchmarking into the Class-level Era" [paper]
2024-11	arXiv	RustRepoTrans	375	C++, Java, Python, Rust	"Repository-level Code Translation Benchmark Targeting Rust" [paper] [data]

Program Repair

• "Neural Program Repair: Systems, Challenges and Solutions", 2022-02, Internetware 2022, [paper]
• "A Survey of Learning-based Automated Program Repair", 2023-01, arXiv, [paper]
• "A Survey on Automated Program Repair Techniques", 2023-03, arXiv, [paper]

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2014-07	ISSTA 2014	Defects4J	357	ชวา	"Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs" [paper] [data]
2015-12	IEEE Trans. Software Engineering	ManyBugs/IntroClass	185/998	C	"The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs" [paper] [data]
2016-11	FSE 2016	BugAID	105K	JS	"Discovering Bug Patterns in JavaScript" [paper] [data]
2017-02	AAAI 2017	DeepFix	6971	C	"DeepFix: Fixing Common C Language Errors by Deep Learning" [paper] [data]
2017-05	ICSE-C 2017	Codeflaws	3902	C	"DeepFix: Fixing Common C Language Errors by Deep Learning" [paper] [data]
2017-10	SPLASH 2017	QuixBugs	80	Java, Python	"QuixBugs: a multi-lingual program repair benchmark set based on the quixey challenge" [paper] [data]
2018-05	MSR 2018	Bugs.jar	1158	ชวา	"Bugs.jar: a large-scale, diverse dataset of real-world Java bugs" [paper] [data]
2018-12	ACM Trans. Softw. อังกฤษ Methodol.	BFP	124K	ชวา	"An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation" [paper] [data]
2019-01	SANER 2019	Bears	251	ชวา	"Bears: An Extensible Java Bug Benchmark for Automatic Program Repair Studies" [paper] [data]
2019-01	ICSE 2019	unnamed	21.8K *	ชวา	"On Learning Meaningful Code Changes via Neural Machine Translation" [paper] [data]
2019-04	ICST 2019	BugsJS	453	JS	"BugsJS: a Benchmark of JavaScript Bugs" [paper] [data]
2019-05	ICSE 2019	BugSwarm	1827/1264	Java/Python	"BugSwarm: mining and continuously growing a dataset of reproducible failures and fixes" [paper] [data]
2019-05	ICSE 2019	CPatMiner	17K *	ชวา	"Graph-based mining of in-the-wild, fine-grained, semantic code change patterns" [paper] [data]
2019-05	MSR 2020	ManySStuBs4J	154K	ชวา	"How Often Do Single-Statement Bugs Occur? The ManySStuBs4J Dataset" [paper] [data]
2019-11	ASE 2019	Refactory	1783	งูหลาม	"Re-factoring based program repair applied to programming assignments" [paper] [data]
2020-07	ISSTA 2020	มะพร้าว	24M	Java, Python, C, JS	"CoCoNuT: combining context-aware neural translation models using ensemble for program repair" [paper] [data]
2020-10	Inf. Softw. Technol.	Review4Repair	58021	ชวา	"Review4Repair: Code Review Aided Automatic Program Repairing" [paper] [data]
2020-11	ESEC/FSE 2020	BugsInPy	493	งูหลาม	"BugsInPy: A Database of Existing Bugs in Python Programs to Enable Controlled Testing and Debugging Studies" [paper] [data]
2021-07	ICML 2021	TFix	105K	JS	"TFix: Learning to Fix Coding Errors with a Text-to-Text Transformer" [paper] [data]
2021-08	arXiv	Megadiff	663K *	ชวา	"Megadiff: A Dataset of 600k Java Source Code Changes Categorized by Diff Size" [paper] [data]
2022-01	SSB/TSSB	MSR 2022	9M/3M	งูหลาม	"TSSB-3M: Mining single statement bugs at massive scale" [paper] [data]
2022-10	MSR 2022	FixJS	324K	JS	"FixJS: a dataset of bug-fixing JavaScript commits" [paper] [data]
2022-11	ESEC/FSE 2022	TypeBugs	93	งูหลาม	"PyTER: Effective Program Repair for Python Type Errors" [paper] [data]
2023-03	arXiv	xCodeEval	4.7M	C, C#, C++, Go, Java, JS, Kotlin, PHP, Python, Ruby, Rust	"xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval" [paper] [data]
2023-04	arXiv	RunBugRun	450K	C, C++, Java, Python, JS, Ruby, Go, PHP	"RunBugRun -- An Executable Dataset for Automated Program Repair" [paper] [data]
2023-08	arXiv	HumanEvalPack	984	Python, JS, Go, Java, C++, Rust	"OctoPack: Instruction Tuning Code Large Language Models" [paper] [data]
2024-01	arXiv	DebugBench	4253	C++, Java, Python	"DebugBench: Evaluating Debugging Capability of Large Language Models" [paper] [data]
2024-11	arXiv	MdEval	3513	18	"MdEval: Massively Multilingual Code Debugging" [paper]

* These are code-change datasest, and only a subset therein concerns bug fixing.

Code Summarization

• "A Survey of Automatic Source Code Summarization", 2022-02, Symmetry, [paper]

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2016-08	ACL 2016	CODE-NN	66K/32K	C#/SQL	"Summarizing Source Code using a Neural Attention Model" [paper] [data]
2017-07	IJCNLP 2017	unnamed	150K	งูหลาม	"A parallel corpus of Python functions and documentation strings for automated code documentation and code generation" [paper] [data]
2018-05	ICPC 2018	DeepCom	588K	ชวา	"Deep code comment generation" [paper] [data]
2018-07	IJCAI 2018	TL-CodeSum	411K	ชวา	"Summarizing Source Code with Transferred API Knowledge" [paper] [data]
2018-11	ASE 2018	unnamed	109K	งูหลาม	"Improving Automatic Source Code Summarization via Deep Reinforcement Learning" [paper] [data]
2019-09	arxiv	CodeSearchNet	2.3M	Go, JS, Python, PHP, Java, Ruby	"CodeSearchNet Challenge: Evaluating the State of Semantic Code Search" [paper] [data]
2023-08	arXiv	HumanEvalPack	984	Python, JS, Go, Java, C++, Rust	"OctoPack: Instruction Tuning Code Large Language Models" [paper] [data]

Defect/Vulnerability Detection

• "Benchmarking Software Vulnerability Detection Techniques: A Survey", 2023-03, arXiv, [paper]

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2018-01	NDSS 2018	CGD	62K	C, C++	"VulDeePecker: A Deep Learning-Based System for Vulnerability Detection" [paper] [data]
2018-04	IEEE Trans. Ind. Informatics	unnamed	32988	C, C++	"Cross-Project Transfer Representation Learning for Vulnerable Function Discovery" [paper] [data]
2018-07	ICMLA 2018	Draper VDISC	12.8M	C, C++	"Automated Vulnerability Detection in Source Code Using Deep Representation Learning" [paper] [data]
2018-07	IEEE TDSC	SySeVR	15591	C, C++	"SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities" [paper] [data]
2019-02	MSR 2019	unnamed	624	ชวา	"A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software" [paper] [data]
2019-09	NeurIPS 2019	Devign	49K	C	"Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks" [paper] [data]
2019-11	IEEE TDSC	unnamed	170K	C, C++	"Software Vulnerability Discovery via Learning Multi-Domain Knowledge Bases" [paper] [data]
2019-12	ICLR 2020	ยอดเยี่ยม	2.8M	งูหลาม	"Global Relational Models of Source Code" [paper] [data]
2020-01	IEEE TDSC	MVD	182K	C, C++	"μVulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection" [paper] [data]
2020-02	ICICS 2019	unnamed	1471	C	"Deep Learning-Based Vulnerable Function Detection: A Benchmark" [paper] [data]
2020-09	IEEE Trans. Software Eng.	เปิดเผย	18K	C	"Deep Learning based Vulnerability Detection: Are We There Yet?" [paper] [data]
2020-09	MSR 2020	Big-Vul	265K	C, C++	"AC/C++ Code Vulnerability Dataset with Code Changes and CVE Summaries" [paper] [data]
2021-02	ICSE (SEIP) 2021	D2A	1.3M	C, C++	"D2A: A Dataset Built for AI-Based Vulnerability Detection Methods Using Differential Analysis" [paper] [data]
2021-05	NeurIPS 2021	PyPIBugs	2374	งูหลาม	"Self-Supervised Bug Detection and Repair" [paper] [data]
2021-07	In PROMISE 2021	CVEfixes	5495	27	"CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Software" [paper] [data]
2021-08	ESEC/FSE 2021	CrossVul	27476	40+	"CrossVul: a cross-language vulnerability dataset with commit data" [paper] [data]
2023-04	RAID 2023	DiverseVul	349K	C, C++	"DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detection" [paper] [data]
2023-06	arXiv	VulnPatchPairs	26K	C	"Limits of Machine Learning for Automatic Vulnerability Detection" [paper] [data]
2023-11	arXiv	VulBench	455	C	"How Far Have We Gone in Vulnerability Detection Using Large Language Models" [paper] [data]
2024-03	arXiv	PrimeVul	236K	C/C++	"Vulnerability Detection with Code Language Models: How Far Are We?" [กระดาษ]
2024-06	arXiv	VulDetectBench	1,000	C/C++	"VulDetectBench: Evaluating the Deep Capability of Vulnerability Detection with Large Language Models" [paper] [data]
2024-08	arXiv	CodeJudge-Eval	1860	งูหลาม	"CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?" [paper] [data]
2024-11	arXiv	CleanVul	11632	Java, Python, JS, C#, C/C++	"CleanVul: Automatic Function-Level Vulnerability Detection in Code Commits Using LLM Heuristics" [paper] [data]

Code Retrieval

• "Code Search: A Survey of Techniques for Finding Code", 2022-04, ICSME 2021, [[paper](ACM Comput. Surv)]
• "A Survey of Deep Code Search", 2023-05, arXiv, [paper]

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2018-03	WWW 2018	StaQC	148K/120K	Python/SQL	"StaQC: A Systematically Mined Question-Code Dataset from Stack Overflow" [paper] [data]
2018-05	ICSE 2018	DeepCS	16.2M	ชวา	"Deep Code Search" [paper] [data]
2018-05	MSR 2018	CoNaLa	600K/2.9K	งูหลาม	"Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflow" [paper] [data]
2019-08	arXiv	unnamed	287	ชวา	"Neural Code Search Evaluation Dataset" [paper] [data]
2019-09	arXiv	CodeSearchNet	2.3M/99	Go, PHP, JS, Python, Java, Ruby	"CodeSearchNet Challenge: Evaluating the State of Semantic Code Search" [paper] [data]
2020-02	SANER 2020	CosBench	52	ชวา	"Are the Code Snippets What We Are Searching for? A Benchmark and an Empirical Study on Code Search with Natural-Language Queries" [paper] [data]
2020-08	arXiv	SO-DS	2.2K	งูหลาม	"Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intent" [paper] [data]
2020-10	ACM Trans. Knowl. Discov. ข้อมูล	FB-Java	249K	ชวา	"Deep Graph Matching and Searching for Semantic Code Retrieval" [paper] [data]
2021-02	NeurIPS Datasets and Benchmarks 2021	AdvTest/WebQueryTest	280K/1K	งูหลาม	"CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation" [paper] [[data]]
2021-05	ACL/IJCNLP 2021	CoSQA	21K	งูหลาม	"CoSQA: 20,000+ Web Queries for Code Search and Question Answering" [paper] [data]
2024-03	arXiv	ProCQA	5.2M	C, C++, Java, Python, Ruby, Lisp, JS, C#, Go, Rust, PHP	"ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Search" [paper] [data]
2024-06	arXiv	CoSQA+	109K	งูหลาม	"CoSQA+: Enhancing Code Search Dataset with Matching Code" [paper] [data]
2024-07	arXiv	CoIR	~2M	14	"CoIR: A Comprehensive Benchmark for Code Information Retrieval Models" [paper] [data]
2024-08	arXiv	SeqCoBench	14.5K	งูหลาม	"What can Large Language Models Capture about Code Functional Equivalence?" [กระดาษ]

Type Inference

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2019-12	ESEC/FSE 2020	TypeWriter OSS	208K	งูหลาม	"TypeWriter: Neural Type Prediction with Search-based Validation" [paper] [data]
2020-04	PLDI 2020	Typilus	252K	งูหลาม	"Typilus: Neural Type Hints" [paper] [data]
2020-04	ICLR 2020	LambdaNet	300 *	ตัวพิมพ์ใหญ่	"LambdaNet: Probabilistic Type Inference using Graph Neural Networks" [paper] [data]
2021-04	MSR 2021	ManyTypes4Py	869K	งูหลาม	"ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference" [paper] [data]
2022-10	MSR 2022	ManyTypes4TypeScript	9.1M	ตัวพิมพ์ใหญ่	"ManyTypes4TypeScript: a comprehensive TypeScript dataset for sequence-based type inference" [paper] [data]
2023-02	ECOOP 2023	TypeWeaver	513 *	ตัวพิมพ์ใหญ่	"Do Machine Learning Models Produce TypeScript Types That Type Check?" [paper] [data]
2023-03	ICLR 2023	BetterTypes4Py/InferTypes4Py	608K/4.6K	งูหลาม	"TypeT5: Seq2seq Type Inference using Static Analysis" [paper] [data]
2023-05	arXiv	OpenTau	744 *	ตัวพิมพ์ใหญ่	"Type Prediction With Program Decomposition and Fill-in-the-Type Training" [paper] [data]

* These are project counts.

Commit Message Generation

• "On the Evaluation of Commit Message Generation Models: An Experimental Study", 2021-07, ICSME 2021, [paper]

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2017-03	ICPC 2017	unnamed	509K	ชวา	"Towards Automatic Generation of Short Summaries of Commits" [paper] [data]
2017-04	ACL 2017	CommitGen	153K	Python, JS, C++, Java	"A Neural Architecture for Generating Natural Language Descriptions from Source Code Changes" [paper] [data]
2017-08	ASE 2017	CommitGen	32K/75K *	ชวา	"Automatically Generating Commit Messages from Diffs using Neural Machine Translation" [paper] [data]
2018-09	ASE 2018	NNGen	27K	ชวา	"Neural-machine-translation-based commit message generation: how far are we?" [paper] [data]
2019-05	MSR 2019	PtrGNCMsg	64.9K	ชวา	"Generating commit messages from diffs using pointer-generator network" [paper] [[data(https://zenodo.org/records/2593787)]]
2019-08	IJCAI 2019	CoDiSum	90.7K	ชวา	"Commit message generation for source code changes" [paper] [data]
2019-12	IEEE Trans. Software Eng.	อะตอม	160K	ชวา	"ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking" [paper] [data]
2021-05	arXiv	CommitBERT	346K	Python, PHP, Go, Java, JS, Ruby	"CommitBERT: Commit Message Generation Using Pre-Trained Programming Language Model" [paper] [data]
2021-07	ICSME 2021	MCMD	2.25M	Java, C#, C++, Python, JS	"On the Evaluation of Commit Message Generation Models: An Experimental Study" [paper] [data]
2021-07	ACM Trans. Softw. อังกฤษ Methodol.	CoRec	107K	ชวา	"Context-aware Retrieval-based Deep Commit Message Generation" [paper] [data]
2023-07	ASE 2023	ExGroFi	19263	ชวา	"Delving into Commit-Issue Correlation to Enhance Commit Message Generation Models" [paper] [data]
2023-08	ASE 2023	CommitChronicle	10.7M	20	"From Commit Message Generation to History-Aware Commit Message Completion" [paper] [data]

* with/without verb-direct object filter

Repo-Level Coding

วันที่	สถานที่จัดงาน	เกณฑ์มาตรฐาน	ขนาด	ภาษา	แหล่งที่มา
2023-03	arXiv	RepoEval	1600/1600/373 *	งูหลาม	"RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation" [paper] [data]
2023-06	ICLR 2024	RepoBench	890K/9M/43K $^dagger$	Python, Java	"RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems" [paper] [data]
2023-06	NeurIPS 2023	PragmaticCode	880 **	ชวา	"Guiding Language Models of Code with Global Context using Monitors" [paper] [data]
2023-06	arXiv	Stack-Repo	816K	ชวา	"RepoFusion: Training Code Models to Understand Your Repository" [paper] [data]
2023-09	ISMB 2024	BioCoder	2269/460/460	Python, Java	"BioCoder: A Benchmark for Bioinformatics Code Generation with Large Language Models" [paper] [data]
2023-09	arXiv	CodePlan	645/21 $^ddagger$	C#/Python $^ddagger$	"CodePlan: Repository-level Coding using LLMs and Planning" [paper] [data]
2023-10	arXiv	SWE-Bench	2294	งูหลาม	"SWE-bench: Can Language Models Resolve Real-World GitHub Issues?" [paper] [data]
2023-10	arXiv	CrossCodeEval	9928	Python, Java, TypeScript, C#	"CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion" [paper] [data]
2024-03	arXiv	EvoCodeBench	275	งูหลาม	"EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories" [paper] [data]
2024-05	ACL 2024 Findings	DevEval	1874	งูหลาม	"DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories" [paper] [data]
2024-06	arXiv	JavaBench	389	ชวา	"Can AI Beat Undergraduates in Entry-level Java Assignments? Benchmarking Large Language Models on JavaBench" [paper] [data]
2024-06	arXiv	HumanEvo	200/200	Python/Java	"Towards more realistic evaluation of LLM-based code generation: an experimental study and beyond" [paper] [data]
2024-06	arXiv	RepoExec	355	งูหลาม	"REPOEXEC: Evaluate Code Generation with a Repository-Level Executable Benchmark" [paper]
2024-06	arXiv	RES-Q	100	Python, JavaScript	"RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale" [paper] [data]
2024-08	arXiv	SWE-bench-java	91	ชวา	"SWE-bench-java: A GitHub Issue Resolving Benchmark for Java" [paper] [data]
2024-10	arXiv	Codev-Bench	296	งูหลาม	"Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion?" [paper] [data]
2024-10	arXiv	SWE-bench M	617	จาวาสคริปต์	"SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?" [paper] [data]
2024-10	arXiv	SWE-Bench+	548	งูหลาม	"SWE-Bench+: Enhanced Coding Benchmark for LLMs" [paper] [data]
2024-10	arXiv	DA-Code	500	Python, Bash, SQL	"DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models" [paper] [data]
2024-10	arXiv	RepoCod	980	งูหลาม	"Can Language Models Replace Programmers? REPOCOD Says 'Not Yet'" [paper]
2024-10	arXiv	M2rc-Eval	5993 repos	18	"M2rc-Eval: Massively Multilingual Repository-level Code Completion Evaluation" [paper] [data]

*Line Completion/API Invocation Completion/Function Completion

$^dagger$ Retrieval/Completion/Pipeline

** File count

$^ddagger$ Migration/Temporal Edit

Other tasks are coming soon!

9. Recommended Readings

30 papers as a primer on LLM.

วันที่	คำสำคัญ	กระดาษ	TL;DR
2014-09	ความสนใจ	Neural Machine Translation by Jointly Learning to Align and Translate	The original attention, proposed for encoder-decoder RNN
2015-08	BPE	Neural Machine Translation of Rare Words with Subword Units	Byte-pair encoding: split rare words into subword units
2017-06	Transformer	Attention Is All You Need	Replace LSTM with self-attention for long-range dependency and parallel training
2017-10	Mixed Precision Training	Mixed Precision Training	Store model weights in fp16 to save memory
2018-04	กาว	GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding	A language understanding benchmark
2018-06	GPT	Improving Language Understanding by Generative Pre-Training	Pretraining-finetuning paradigm applied to Transformer decoder
2018-10	BERT	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	Masked Language Modeling (MLM) applied to Transformer encoder for pretraining
2019-02	GPT-2	Language Models are Unsupervised Multitask Learners	GPT made larger (1.5B). They found language models implicitly learn about downstream tasks (such as translation) during pretraining.
2019-05	SuperGLUE	SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems	Another language understanding benchmark
2019-07	RoBERTa	RoBERTa: A Robustly Optimized BERT Pretraining Approach	An optimized BERT
2019-09	Megatron-LM	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism	Model parallelism
2019-10	ศูนย์	ZeRO: Memory Optimizations Toward Training Trillion Parameter Models	Memory-efficient distributed optimization
2019-10	T5	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	Transformer encoder-decoder pretrained with an MLM-like denoising objective
2020-05	GPT-3	Language Models are Few-Shot Learners	By training an even larger version of GPT-2 (175B), they discovered a new learning paradigm: In-Context Learning (ICL)
2020-09	MMLU	Measuring Massive Multitask Language Understanding	A world-knowledge and complex reasoning benchmark
2020-12	กอง	The Pile: An 800GB Dataset of Diverse Text for Language Modeling	A diverse pretraining dataset
2021-06	LoRA	LORA: การปรับระดับต่ำของแบบจำลองภาษาขนาดใหญ่	Memory-efficient finetuning
2021-09	FLAN	Finetuned Language Models Are Zero-Shot Learners	Instruction-finetuning
2021-10	T0	Multitask Prompted Training Enables Zero-Shot Task Generalization	Also instruction finetuning, but applied to the much smaller T5
2021-12	Gopher	Scaling Language Models: Methods, Analysis & Insights from Training Gopher	A 280B LLM with comprehensive experiments
2022-01	CoT	Chain-of-Thought Prompting Elicits Reasoning in Large Language Models	Chain-of-Though reasoning
2022-03	InstructGPT	Training language models to follow instructions with human feedback	GPT-3 instruction finetuned with RLHF (reinforcement learning from human feedback)
2022-03	Chinchilla	Training Compute-Optimal Large Language Models	A smaller (70B) version of Gopher that's pretrained on more data
2022-04	ปาล์ม	PaLM: Scaling Language Modeling with Pathways	The largest dense model ever (540B)
2022-05	0-shot CoT	Large Language Models are Zero-Shot Reasoners	Tell LLMs to think step by step, and they can actually do it
2022-06	BIG Bench	Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models	Another world-knowledge and complex reasoning benchmark
2022-06	Emergent Ability	Emergent Abilities of Large Language Models	A review on emergent abilities
2022-10	Flan	Scaling Instruction-Finetuned Language Models	Consolidate all the existing instruction tuning datasets, and you get SOTA
2022-11	BLOOM	BLOOM: A 176B-Parameter Open-Access Multilingual Language Model	The largest open-source LLM, trained on 46 languages, with detailed discussion about training and evaluation
2022-12	Self-Instruct	Self-Instruct: Aligning Language Models with Self-Generated Instructions	Instruction tuning using LLM-generated data

This list aims to provide the essential background for understanding current LLM technologies, and thus excludes more recent models such as LLaMA, GPT-4 or PaLM 2. For comprehensive reviews on these more general topics, we refer to other sources such as this paper or these repositories: Awesome-LLM, Awesome AIGC Tutorials. And for LLM applications in other specific domains: Awesome Domain LLM, Awesome Tool Learning, Awesome-LLM-MT, Awesome Education LLM.

การอ้างอิง

If you find this repo or our survey helpful, please consider citing us:

 @article{zhang2024unifying,
   title={Unifying the Perspectives of {NLP} and Software Engineering: A Survey on Language Models for Code},
   author={Ziyin Zhang and Chaoyu Chen and Bingchang Liu and Cong Liao and Zi Gong and Hang Yu and Jianguo Li and Rui Wang},
   journal={Transactions on Machine Learning Research},
   issn={2835-8856},
   year={2024},
   url={https://openreview.net/forum?id=hkNnGqZnpa},
   note={}
}

ประวัติดาว

Join US

การรับสมัคร

English version

We are the AI Native team within the Platform Technology Business Group at Ant Group, dedicated to the intelligentization of Ant Group's platform engineering. Established for over three years, our team has played a pivotal role in supporting the intelligent operation and maintenance of Ant Group's cloud computing infrastructure. Our mission is to build algorithm services and platforms with a wide user base through world-class technological innovation and impact, supporting the implementation of internal and external products and businesses. Embracing an innovation-driven ethos, our team not only supports business implementation but also propels technological influence. Over the past three years, we have published more than 20 papers at top conferences like ICLR, NeurIPS, KDD, and ACL. Our innovative business outcomes have earned us two Ant Technology's highest T-Star awards and one SuperMA award from Ant Group. Our open-source project CodeFuse has received 4K stars as of February 2024, and our models have been downloaded over 1.5 million times on Huggingface and Modelscope.

We are on the lookout for top talents to join our vibrant team! If you're eager to develop your career in an environment filled with energy, innovation, and a culture of excellence, we welcome you to explore our career opportunities for both campus and experienced hires. Join us and be a part of creating the next milestone in the industry.

Campus Recruitment : https://hrrecommend.antgroup.com/guide.html?code=8uoP5mlus5DqQYbE_EnqcE2FD5JZH21MwvMUIb9mb6X3osXPuBraG54SyM8GLn_7

Experienced Hires : https://talent.antgroup.com/off-campus-position?positionId=1933830

中文版

我们是平台技术事业群 AI Native 团队，负责蚂蚁蚂蚁集团平台工程的智能化，团队成立 3 年多以来，支持了蚂蚁集团云计算基础设施智能化运维的升级改造。团队的 Mission 是，通过世界级的技术创新和影响，构建有广泛用户的算法服务和平台，支撑内外部产品和业务落地。团队秉承创新基因，在支撑业务落地的同时，推动技术影响。3 年以来在 ICLR、NeurIPS、KDD、ACL 等顶会发表论文 20 余篇，创新业务结果获得两次蚂蚁技术最高奖 T-Star，1 次蚂蚁集团最高奖 SuperMA。开源项目 CodeFuse 获得 4K 点赞(2024 年 2 月)，Huggingface 和 modelscope 上模型累积下载量超过 150 万次。

我们正在寻找行业中的佼佼者加入我们的团队！如果您希望在一个充满活力、创新和卓越文化的环境中发展您的职业生涯，欢迎您查看我们的社招&校招机会，加入我们，一起创造下一个行业里程碑。

校招：https://hrrecommend.antgroup.com/guide.html?code=8uoP5mlus5DqQYbE_EnqcE2FD5JZH21MwvMUIb9mb6X3osXPuBraG54SyM8GLn_7

社招：https://talent.antgroup.com/off-campus-position?positionId=1933830

Contact Us (WeChat)

永久企业微信交流群：