이 저장소는 사용자의 프롬프트와 해당 단위 테스트를 수신하는 LLM 에이전트를 구현합니다. 에이전트는 코드를 생성하고 올바른 실행을 테스트하며 단위 테스트를 통과하는지 확인합니다. 코드가 실행되지 않거나 단위 테스트에 실패하면 에이전트는 성공할 때까지 코드를 다시 생성합니다.
이 프로젝트는 LangChain의 노트북(https://github.com/mistralai/cookbook/blob/main/third_party/langchain/langgraph_code_assistant_mistral.ipynb)의 코드를 기반으로 구축되었으며, 여기서 에이전트는 사용자의 프롬프트를 기반으로 코드를 생성하고 실행 여부를 확인합니다. 공장.
이 프로젝트의 주요 기여는 단위 테스트를 코드 실행 프로세스에 통합하는 것입니다.
또한 원래 Jupyter 노트북 코드는 배포에 적합한 구조화된 Python 프로젝트로 리팩터링되었습니다. 최고의 코딩 방법을 준수함으로써 코드가 모듈화되고, 재사용 가능한 기능이 생성되었으며, 적절한 문서화 및 테스트가 보장되었습니다. 그 결과, 유지 관리가 가능하고 프로덕션 준비가 완료된 잘 구성된 .py 프로젝트가 탄생했습니다.
git clone https : // github . com / paulomuraroferreira / Self_correcting_coding_agent . git
cd Self_correcting_coding_agent
$ pip install - e .
.env 파일을 만들고 다음 환경 변수를 입력합니다.
OPENAI_API_KEY = your_openai_api_key
OPENAI_CHAT_MODEL = "gpt-4o-2024-08-06"
main.py 파일에 코드 프롬프트를 입력합니다. 프롬프트는 설명적이어야 하며 클래스 이름을 지정해야 합니다.
예를 들어:
QUESTION3 = '''
Python Class Description
Class Name: BankAccount
Description:
The BankAccount class represents a user's bank account, allowing for deposits, withdrawals, and viewing the transaction history. The class ensures that withdrawals cannot exceed the current balance and that deposits and withdrawals are properly recorded in the transaction history.
Attributes:
balance (float): The current balance of the account, initialized to 0.
transactions (list): A list to store the history of transactions. Each transaction is stored as a dictionary with keys type (either 'deposit' or 'withdrawal'), amount, and date.
Methods:
deposit(amount: float) -> None: Adds the specified amount to the balance and records the transaction.
withdraw(amount: float) -> bool: Attempts to subtract the specified amount from the balance. Returns True if successful, otherwise returns False. Records the transaction if successful.
get_balance() -> float: Returns the current balance.
get_transaction_history() -> list: Returns a list of all transactions.
'''
main = Main(QUESTION3)
src/unit_test_folder의 .py 파일에 단위 테스트 코드를 추가합니다. 또한 code_solution.py 파일에서 해당 클래스를 가져옵니다. 예를 들어 위 프롬프트의 경우 LLM은 BankAccount라는 클래스를 생성합니다. 따라서 src/unit_test_folder/test_unit_test.py에는 다음이 포함되어야 합니다.
src.code_solution에서 BankAccount 가져오기
main.py 파일을 실행합니다.
python main . py
위의 예에서 코드는 다음 출력을 표시합니다.
================================ Human Message =================================
Python Class Description
Class Name: BankAccount
Description:
The BankAccount class represents a user's bank account, allowing for deposits, withdrawals, and viewing the transaction history. The class ensures that withdrawals cannot exceed the current balance and that deposits and withdrawals are properly recorded in the transaction history.
Attributes:
balance (float): The current balance of the account, initialized to 0.
transactions (list): A list to store the history of transactions. Each transaction is stored as a dictionary with keys type (either 'deposit' or 'withdrawal'), amount, and date.
Methods:
deposit(amount: float) -> None: Adds the specified amount to the balance and records the transaction.
withdraw(amount: float) -> bool: Attempts to subtract the specified amount from the balance. Returns True if successful, otherwise returns False. Records the transaction if successful.
get_balance() -> float: Returns the current balance.
get_transaction_history() -> list: Returns a list of all transactions.
2024-08-18 14:51:24,955 - INFO - agent.py - ---GENERATING CODE SOLUTION---
================================== Ai Message ==================================
Here is my attempt to solve the problem: The BankAccount class is designed to manage a user's bank account by allowing deposits, withdrawals, and tracking transaction history. It ensures that withdrawals do not exceed the available balance and records each transaction with details such as type, amount, and date.
Imports: from datetime import datetime
Code: class BankAccount:
def __init__(self):
self.balance = 0.0
self.transactions = []
def deposit(self, amount: float) -> None:
self.balance += amount
transaction = {
'type': 'deposit',
'amount': amount,
'date': datetime.now()
}
self.transactions.append(transaction)
def withdraw(self, amount: float) -> bool:
if amount > self.balance:
return False
self.balance -= amount
transaction = {
'type': 'withdrawal',
'amount': amount,
'date': datetime.now()
}
self.transactions.append(transaction)
return True
def get_balance(self) -> float:
return self.balance
def get_transaction_history(self) -> list:
return self.transactions
2024-08-18 14:51:28,979 - INFO - agent.py - ---CHECKING CODE---
2024-08-18 14:51:29,170 - INFO - agent.py -
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.3.2, pluggy-1.5.0 -- /home/paulo/Python_projects/Self_correcting_coding_agent/venv/bin/python
cachedir: .pytest_cache
rootdir: /home/paulo/Python_projects/Self_correcting_coding_agent
configfile: pyproject.toml
plugins: anyio-4.4.0
collecting ... collected 7 items
src/unit_test_folder/test_unit_test.py::test_initial_balance PASSED [ 14%]
src/unit_test_folder/test_unit_test.py::test_deposit PASSED [ 28%]
src/unit_test_folder/test_unit_test.py::test_withdrawal_successful PASSED [ 42%]
src/unit_test_folder/test_unit_test.py::test_withdrawal_insufficient_funds PASSED [ 57%]
src/unit_test_folder/test_unit_test.py::test_transaction_history PASSED [ 71%]
src/unit_test_folder/test_unit_test.py::test_invalid_deposit FAILED [ 85%]
src/unit_test_folder/test_unit_test.py::test_invalid_withdrawal FAILED [100%]
=================================== FAILURES ===================================
_____________________________ test_invalid_deposit _____________________________
account = <src.code_solution.BankAccount object at 0x7fac0d4f0b90>
def test_invalid_deposit(account):
"""Test that depositing a negative amount raises ValueError."""
> with pytest.raises(ValueError):
E Failed: DID NOT RAISE <class 'ValueError'>
src/unit_test_folder/test_unit_test.py:48: Failed
___________________________ test_invalid_withdrawal ____________________________
account = <src.code_solution.BankAccount object at 0x7fac0d4f1f40>
def test_invalid_withdrawal(account):
"""Test that withdrawing a negative amount raises ValueError."""
> with pytest.raises(ValueError):
E Failed: DID NOT RAISE <class 'ValueError'>
src/unit_test_folder/test_unit_test.py:53: Failed
=========================== short test summary info ============================
FAILED src/unit_test_folder/test_unit_test.py::test_invalid_deposit - Failed:...
FAILED src/unit_test_folder/test_unit_test.py::test_invalid_withdrawal - Fail...
========================= 2 failed, 5 passed in 0.02s ==========================
2024-08-18 14:51:29,170 - INFO - agent.py - Some tests failed.
2024-08-18 14:51:29,172 - INFO - agent.py - ---DECISION: RE-TRY SOLUTION---
================================ Human Message =================================
Your solution failed the unit test: ============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.3.2, pluggy-1.5.0 -- /home/paulo/Python_projects/Self_correcting_coding_agent/venv/bin/python
cachedir: .pytest_cache
rootdir: /home/paulo/Python_projects/Self_correcting_coding_agent
configfile: pyproject.toml
plugins: anyio-4.4.0
collecting ... collected 7 items
src/unit_test_folder/test_unit_test.py::test_initial_balance PASSED [ 14%]
src/unit_test_folder/test_unit_test.py::test_deposit PASSED [ 28%]
src/unit_test_folder/test_unit_test.py::test_withdrawal_successful PASSED [ 42%]
src/unit_test_folder/test_unit_test.py::test_withdrawal_insufficient_funds PASSED [ 57%]
src/unit_test_folder/test_unit_test.py::test_transaction_history PASSED [ 71%]
src/unit_test_folder/test_unit_test.py::test_invalid_deposit FAILED [ 85%]
src/unit_test_folder/test_unit_test.py::test_invalid_withdrawal FAILED [100%]
=================================== FAILURES ===================================
_____________________________ test_invalid_deposit _____________________________
account = <src.code_solution.BankAccount object at 0x7fac0d4f0b90>
def test_invalid_deposit(account):
"""Test that depositing a negative amount raises ValueError."""
> with pytest.raises(ValueError):
E Failed: DID NOT RAISE <c ... (truncated)
2024-08-18 14:51:29,175 - INFO - agent.py - ---GENERATING CODE SOLUTION---
================================== Ai Message ==================================
Here is my attempt to solve the problem: The previous solution did not handle cases where negative amounts were deposited or withdrawn. The unit tests expect a ValueError to be raised in such cases, but the original implementation did not include this validation. To fix this, we need to add checks in the deposit and withdraw methods to raise a ValueError if the amount is negative.
Imports: from datetime import datetime
Code: class BankAccount:
def __init__(self):
self.balance = 0.0
self.transactions = []
def deposit(self, amount: float) -> None:
if amount < 0:
raise ValueError("Deposit amount must be positive.")
self.balance += amount
transaction = {
'type': 'deposit',
'amount': amount,
'date': datetime.now()
}
self.transactions.append(transaction)
def withdraw(self, amount: float) -> bool:
if amount < 0:
raise ValueError("Withdrawal amount must be positive.")
if amount > self.balance:
return False
self.balance -= amount
transaction = {
'type': 'withdrawal',
'amount': amount,
'date': datetime.now()
}
self.transactions.append(transaction)
return True
def get_balance(self) -> float:
return self.balance
def get_transaction_history(self) -> ... (truncated)
2024-08-18 14:51:34,913 - INFO - agent.py - ---CHECKING CODE---
2024-08-18 14:51:35,131 - INFO - agent.py -
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.3.2, pluggy-1.5.0 -- /home/paulo/Python_projects/Self_correcting_coding_agent/venv/bin/python
cachedir: .pytest_cache
rootdir: /home/paulo/Python_projects/Self_correcting_coding_agent
configfile: pyproject.toml
plugins: anyio-4.4.0
collecting ... collected 7 items
src/unit_test_folder/test_unit_test.py::test_initial_balance PASSED [ 14%]
src/unit_test_folder/test_unit_test.py::test_deposit PASSED [ 28%]
src/unit_test_folder/test_unit_test.py::test_withdrawal_successful PASSED [ 42%]
src/unit_test_folder/test_unit_test.py::test_withdrawal_insufficient_funds PASSED [ 57%]
src/unit_test_folder/test_unit_test.py::test_transaction_history PASSED [ 71%]
src/unit_test_folder/test_unit_test.py::test_invalid_deposit PASSED [ 85%]
src/unit_test_folder/test_unit_test.py::test_invalid_withdrawal PASSED [100%]
============================== 7 passed in 0.01s ===============================
2024-08-18 14:51:35,131 - INFO - agent.py - All tests passed!
2024-08-18 14:51:35,131 - INFO - agent.py - ---NO CODE TEST FAILURES---
2024-08-18 14:51:35,133 - INFO - agent.py - ---DECISION: FINISH---