Today, we’re open-sourcing Docent under an Apache 2.0 license. Check out our public codebase to self-host Docent, peek under the hood, or open issues & pull requests! As always, the hosted version remains available on our servers.
We’re excited for developers to build upon Docent and integrate it with other tools. If you would like to make contributions, provide feedback, or offer feature requests, please join our community Slack.
What is Docent?
Docent is our tool for understanding complex behaviors in AI agent transcripts. It enables users to explore large collections of transcripts for interesting or unwanted behaviors, and precisely measure how often particular behaviors occur.
For instance, a user might start with a vague question like “is my model reward hacking.” Docent first helps the user explore potential instances of reward hacking in the data. Then, it interactively converts the underspecified question into a precise behavior rubric. Finally, it runs the rubric to provide a precise measurement of how often the behavior occurs.
You can learn more about Docent at our landing page, or read the quickstart guide.
Why we’re open sourcing Docent
At Transluce, one of our key goals is to help set public standards for understanding and overseeing AI systems. We do this by building high-quality technology that is battle-tested across many different frontier AI systems and use cases. This technology then becomes part of the standard toolkit for understanding and oversight.
As part of this, it is important for our technology to be publicly vetted. That way, researchers across the world can help us improve the technology by pointing out flaws and suggesting improvements. Open-sourcing Docent makes this possible.
Additionally, part of our mission is to support the broader ecosystem of AI evaluators. We hope that open sourcing Docent helps other evaluators, across governments, third-party non-profits, academia, and AI companies more easily use and build upon it.