Show HN: CATT 0.2.1: Test agents/services against a Muse/Instinct-alike, in CI
来源:Hacker News 首页收录:2026/10/22
✦ AI 解读
CATT 0.2.1 是开源消费者代理测试工具,帮助团队在 CI 中针对类似 Muse/Instinct 的专有助手测试服务。它通过模拟代理(含记忆和账户状态)解决可重复性难题,允许定义初始条件、审批和成功条件,并记录完整交互(含工具调用与响应)以便对比差异。
Hey HN, we recently open sourced our Consumer Agent Testing Tool to help teams iterate against proprietary assistants like Muse and Instinct. We built this to reduce our own frustrations with building an MCP/API for these agents to consume.The problem: testing against these tools is hard! Agent memory + account statefulness make reproducibility, in the ways required for repeatable/reliable outcomes, between agents and the APIs/MCPs they interact with very challenging.The solution: CATT lets devs test their services against a surrogate Muse or Instinct agent (not affiliated with either of these companies). Testers define starting conditions, approvals, and success conditions for their service's state. CATT then records the full interaction (incl. tool calls, responses) so it can diff what happened from what the agent reports. Runs can be tested in local builds or via CI, with or without LLM backing.CATT is designed to emulate the important parts of personal assistant interactions. Its got:- Tool calls and command-line tool access
- Simulated approvals for service actions
- Outcomes tracking and comparison
- Optional MCP profile based on an unofficial Muse templateThere are limitations to how much we know about different assistant specific behaviors so the tool labels its assumptions as `documented`, `unofficial`, `assumed` or `unknown` for developer review. We'll do our best to keep the tool updated as new insights about the internal workings of these assistants come to light. It is an area of active research and interest, a good example is this recent report on how Instinct uses Exa for web search [0].Are you building a connector or MCP? If yes, we'd like to hear from you! Have you had assistants behave incoherently, use stale information, or procalim success without evidence? What else should we add tests for?The best way to get started is...to point your coding assistant at the repo and ask it to integrate the tool with your CI or local testing pipeline!Hope it's
相关推荐
H2oai/jdupes: H2O.ai fork of jdupes
✦ AI 摘要H2O.ai 在 GitHub 上发布了 jdupes 的分支版本。jdupes 是一个用于查找和去重重复文件的命令行工具,H2O.ai 的分支可能包含针对其内部需求的修改或增强。该项目在 Hacker News 上仅获得 1 分且无评论,关注度较低,但对需要文件去重功能的开发者可能有一定参考价值。
Open Source Mobile-Use MCP and AI Agent
✦ AI 摘要这是一个在 Hacker News 上分享的开源项目 buddy-android,提供移动端使用的 MCP 和 AI Agent 实现。项目托管于 GitHub,目前关注度较低(1 分、1 条评论),但为在 Android 设备上构建和运行 AI 智能体提供了开源参考,适合对移动端 Agent 和 MCP 协议感兴趣的开发者研究。
Show HN: Koala – Run AI Agents in Isolated Linux VMs on macOS
✦ AI 摘要Koala 是一个在 Hacker News 上展示的开源项目,旨在让用户在 macOS 上通过隔离的 Linux 虚拟机运行 AI 智能体。它提供了安全沙箱环境,避免智能体直接访问宿主机系统,适合需要隔离执行 AI 代理任务的开发者。项目通过 Homebrew 分发,目前关注度较低。