autospec
1.0.0
代码入口点
Autospec 是一个端到端测试/QA 代理，使用视觉和文本语言模型来探索和生成 Web 应用程序的常识测试规范。它的目的是在每次交互后模仿用户对整个 UI 输出的判断，以决定是否引发有关应用程序行为的错误，而不是捕获针对严格定义的先前行为的回归。
在 TodoMVC（一个经典的示例 Web 应用程序）上生成并运行 10 个规范：
npx autospecai --url https://todomvc.com/examples/react/dist/ --apikey YOUR_OPENAI_API_KEY
您需要说“是”才能安装 autospecai 软件包，第一次运行可能需要几分钟才能下载依赖项，例如用于执行测试环境的浏览器二进制文件。
运行完成后，您将看到已运行的测试的摘要以及它们是通过还是失败。
成功的规格将保存在工作目录的trajectories文件夹中。您可以随时通过运行以下命令重新执行这些测试：
npx playwright test
根据您现有的 Playwright 设置，您可能需要将“轨迹”添加到 playwright.config.js 文件中的 testDir 中。
复制示例 .env 文件，并在运行应用程序之前填写 OPENAI_API_KEY 或 GOOGLE_GENERATIVE_AI_API_KEY：
mv .env.example .env # rename the example to .env
nano .env # edit as you like > npx autospecai --help
Usage: npx autospecai --url < url > [--model < model > ] [--spec_limit < limit > ] [--help | -h ]
Required:
--url < url > The target URL to run the autospec tests against.
Optional:
--help, -h Show this help message.
--spec_limit < limit > The max number of specs to generate. Default 10.
--model < model > The model to use for spec generation
* " gpt-4o " (default)
* " gemini-1.5-flash-latest "
* " claude-3-haiku "
* (note: Gemini flash is free up to rate limits)
--apikey < key > The relevant API key for the chosen model ' s API.
* If not specified, we ' ll fall back on the
following environment variables:
* OPENAI_API_KEY
* GOOGLE_GENERATIVE_AI_API_KEY
* ANTHROPIC_API_KEY
日志像这样打印到控制台，以及视频、png 和 HTML/JSON DOM 快照旁边的 trajectories/** 文件夹：
Test Summary:
✔ 1. Check if the page loads successfully
1.1) action: markSpecAsComplete, reason: The spec passed, fullProseExplanationOfReasoning100charmax: The page loaded successfully with all expected elements visible., planningThoughtAboutTheActionIWillTake: The page has loaded successfully as indicated by the presence of the header, input field, and footer. I will mark the spec as complete.
✔ 2. Check if the title 'todos' is displayed
2.1) action: markSpecAsComplete, reason: The spec passed, fullProseExplanationOfReasoning100charmax: The title 'todos' is displayed as an h1 element in the header section., planningThoughtAboutTheActionIWillTake: I will verify if the title 'todos' is displayed by checking the presence of the h1 element with the text 'todos'.
✔ 3. Check if the input box with placeholder 'What needs to be done?' is present
3.1) action: markSpecAsComplete, reason: The spec passed, fullProseExplanationOfReasoning100charmax: The input box with the placeholder 'What needs to be done?' is present in the HTML snapshot., planningThoughtAboutTheActionIWillTake: I will check if the input box with the placeholder 'What needs to be done?' is present in the HTML snapshot.
✔ 4. Check if a new todo item can be added
4.1) action: clickOn, cssSelector: input.new-todo, nth: 0, planningThoughtAboutTheActionIWillTake: I will move the mouse cursor to the input field to add a new todo item.
4.2) action: keyboardInputString, cssSelector: input.new-todo, nth: 0, string: Buy groceries, planningThoughtAboutTheActionIWillTake: I will type a new todo item into the input field.
4.3) action: keyboardInputSingleKey, cssSelector: input.new-todo, nth: 0, key: Enter, planningThoughtAboutTheActionIWillTake: I will press the Enter key to add the new todo item.
4.4) action: markSpecAsComplete, reason: The spec passed, fullProseExplanationOfReasoning100charmax: The new todo item 'Buy groceries' was successfully added to the list., planningThoughtAboutTheActionIWillTake: I will mark the spec as complete because the new todo item 'Buy groceries' has been successfully added.
✔ 5. Check if a todo item can be marked as completed
5.1) action: keyboardInputString, cssSelector: input.new-todo, nth: 0, string: Test Todo Item, planningThoughtAboutTheActionIWillTake: To check if a todo item can be marked as completed, I first need to add a new todo item. I will type a new todo item in the input field and press Enter.
5.2) action: keyboardInputSingleKey, cssSelector: input.new-todo, nth: 0, key: Enter, planningThoughtAboutTheActionIWillTake: I need to press Enter to add the new todo item to the list.
5.3) action: clickOn, cssSelector: input.toggle, nth: 0, planningThoughtAboutTheActionIWillTake: Now that the todo item is added, I will mark it as completed by clicking the checkbox next to the todo item.
5.4) action: markSpecAsComplete, reason: The spec passed, fullProseExplanationOfReasoning100charmax: The todo item was successfully marked as completed., planningThoughtAboutTheActionIWillTake: The todo item has been marked as completed, as indicated by the strikethrough text and the checkbox being checked. I will mark the spec as complete.
...
autospec passed for https://todomvc.com/examples/react/dist/
autospec 是开源的，我们欢迎贡献者！请打开问题或拉取请求以开始。
该项目已获得 MIT 许可证的许可。有关详细信息，请参阅许可证文件。
