Introduction
Foundational model post-training saw a significant change in 2025. When the first reasoning model o1 came out in late 2024, most of the post-training was done with RLHF. Labs were using an army of contractors that were manually labelling LLM responses based on specific criteria, for example, for mathematicians labelling was based on output and method correctness, for linguists it was based on more subjective creative writing criteria etc. Regardless of the domain, contractors were presented with 3 LLM output examples and chose the best answer. This was then fed back to the LLM in post-training, and the next iteration of the model would be more aligned with human labelling preferences. This was repeated a few times until the model was released.
In 2025 we saw a significant shift in post-training - reinforcement learning with verifiable reward (RLVR). Instead of a human choosing the preferred answer, a verification function determines if the answer is correct - for example, do the unit tests pass, or does the math check out. This works best in verifiable domains such as maths and coding, which are exactly the domains which have improved the most in recent months.
Alongside getting better at coding / maths, these models became a lot better at agentic tasks too - pursuing the task completion in agentic loop until it is solved. This is the most obvious in coding agents that are with great determination pursuing implementing the feature and running tests/linter/typecheck are important checkpoints that help the agent know if they are on the right track. As a side effect the latest LLMs and coding agents got very good at pursuing verifiable milestones.
Building for Verifiability
Andrej Karpathy built autoresearch - AI agent that improves his other project, nanochat, by doing optimisations, retraining the neural net from scratch, and observing the results of the new training run. It managed to optimise nanochat well enough to reduce time to GPT-2 (time it takes to train ~equivalent of GPT-2 from 2019) to 1.8 hours, 0.22 hour improvement over his most optimised human-tuned version beforehand. (As of 14/3/2026 he actually pushed another 0.15 hour improvement).
This is a fantastic example of how existing coding agents are exceptionally good at pursuing the goal as long as the output correctness is verifiable.
We've built our own LLM observability system including platform to run evaluation functions. It allows us to trace our AI agents in production and optimise them in predictable way with contained regressions. When we encounter new edge case in production that we didn't handle well before, we simply add new example into eval dataset, then update the relevant prompts, run the eval and finally see if results are better or worse. If they are worse we do some more prompt engineering until they improve.
In other words this process is fully verifiable and therefore plays to the strength of modern agentic systems. We've built tooling to further piggyback on current coding agents strengths, for example, we've automated human prompt engineering by having a claude skill that researches AI agent problem in production, adds relevant examples to dataset (or updates existing examples), runs the eval, observes the results, updates the relevant prompts & reruns the evals again in the loop until the eval output improves.
It works remarkably well, and now that we are running more than 20 AI agents in production, it's an essential way for us to maintain them. We are building more and more tooling where outcome is verifiable and hence, improvable by modern AI agents.
Building AI Native Company
If we extrapolate, then building a company is verifiable. Company takes input parameters such as people and cash and output is business making money.
We are not yet in the stage where building entire company is doable by AI agents (although it should be possible in future), but given it is possible in the limit, there have to be building blocks that make up the company that are verifiable. We believe that to be a truly AI native company we need to maximise verifiability within business functions and remove unnecessary human elements such as reviews or approvals. At Briefcase we have three core competencies: Product, Customer support and GTM.
The more each business function becomes AI native, the more we as a company become AI native. Some functions are foundational and support the other ones, such as AI product development supports how we do customer support.
Our product is AI native, after all we are automating accounting workflows with our AI agents. The way we develop product is also AI native: our projects are owned end to end by a single person (removing unnecessary human communication steps), we do no human code reviews (removing unnecessary human in the loop step), we have automated alerts based on production issues, we have claude skills that debug those issues and run claude code to fix them too.
The way we do customer support is also AI native: we've developed a series of claude skills to debug customer issue, fix the issue if it's a bug and draft the reply back to the customer.
Our GTM is AI native too, we have qualification agent that qualifies each lead, we have claude skill that runs AEO (Answer Engine Optimization, SEO for AI search engines) optimisation, we have series of claude skills that manage data in our CRM, another one that drafts emails or researches the status of the deal.
More and more of our usages of AI are verifiable such as alert resolution or AEO optimisation. This means that they can already be done with very little human input and as the models / AI agent harnesses become even better, there is a path towards having these workflows automated end to end. As an AI native company, it is one of our building principles that we as a company have to get better as AI ecosystem gets better, and building for verifiability means we build for AI agent optimisation, directly benefiting from latest AI ecosystem advancements.
Build with Us
We are just getting started, there is so much more we want to build across all our core business functions. If you are any sort of builder (engineer, GTM, design), join us, you will get to build the most interesting product & tooling and advance the accounting industry forward in leaps.
Open roles
Introduction
Foundational model post-training saw a significant change in 2025. When the first reasoning model o1 came out in late 2024, most of the post-training was done with RLHF. Labs were using an army of contractors that were manually labelling LLM responses based on specific criteria, for example, for mathematicians labelling was based on output and method correctness, for linguists it was based on more subjective creative writing criteria etc. Regardless of the domain, contractors were presented with 3 LLM output examples and chose the best answer. This was then fed back to the LLM in post-training, and the next iteration of the model would be more aligned with human labelling preferences. This was repeated a few times until the model was released.
In 2025 we saw a significant shift in post-training - reinforcement learning with verifiable reward (RLVR). Instead of a human choosing the preferred answer, a verification function determines if the answer is correct - for example, do the unit tests pass, or does the math check out. This works best in verifiable domains such as maths and coding, which are exactly the domains which have improved the most in recent months.
Alongside getting better at coding / maths, these models became a lot better at agentic tasks too - pursuing the task completion in agentic loop until it is solved. This is the most obvious in coding agents that are with great determination pursuing implementing the feature and running tests/linter/typecheck are important checkpoints that help the agent know if they are on the right track. As a side effect the latest LLMs and coding agents got very good at pursuing verifiable milestones.
Building for Verifiability
Andrej Karpathy built autoresearch - AI agent that improves his other project, nanochat, by doing optimisations, retraining the neural net from scratch, and observing the results of the new training run. It managed to optimise nanochat well enough to reduce time to GPT-2 (time it takes to train ~equivalent of GPT-2 from 2019) to 1.8 hours, 0.22 hour improvement over his most optimised human-tuned version beforehand. (As of 14/3/2026 he actually pushed another 0.15 hour improvement).
This is a fantastic example of how existing coding agents are exceptionally good at pursuing the goal as long as the output correctness is verifiable.
We've built our own LLM observability system including platform to run evaluation functions. It allows us to trace our AI agents in production and optimise them in predictable way with contained regressions. When we encounter new edge case in production that we didn't handle well before, we simply add new example into eval dataset, then update the relevant prompts, run the eval and finally see if results are better or worse. If they are worse we do some more prompt engineering until they improve.
In other words this process is fully verifiable and therefore plays to the strength of modern agentic systems. We've built tooling to further piggyback on current coding agents strengths, for example, we've automated human prompt engineering by having a claude skill that researches AI agent problem in production, adds relevant examples to dataset (or updates existing examples), runs the eval, observes the results, updates the relevant prompts & reruns the evals again in the loop until the eval output improves.
It works remarkably well, and now that we are running more than 20 AI agents in production, it's an essential way for us to maintain them. We are building more and more tooling where outcome is verifiable and hence, improvable by modern AI agents.
Building AI Native Company
If we extrapolate, then building a company is verifiable. Company takes input parameters such as people and cash and output is business making money.
We are not yet in the stage where building entire company is doable by AI agents (although it should be possible in future), but given it is possible in the limit, there have to be building blocks that make up the company that are verifiable. We believe that to be a truly AI native company we need to maximise verifiability within business functions and remove unnecessary human elements such as reviews or approvals. At Briefcase we have three core competencies: Product, Customer support and GTM.
The more each business function becomes AI native, the more we as a company become AI native. Some functions are foundational and support the other ones, such as AI product development supports how we do customer support.
Our product is AI native, after all we are automating accounting workflows with our AI agents. The way we develop product is also AI native: our projects are owned end to end by a single person (removing unnecessary human communication steps), we do no human code reviews (removing unnecessary human in the loop step), we have automated alerts based on production issues, we have claude skills that debug those issues and run claude code to fix them too.
The way we do customer support is also AI native: we've developed a series of claude skills to debug customer issue, fix the issue if it's a bug and draft the reply back to the customer.
Our GTM is AI native too, we have qualification agent that qualifies each lead, we have claude skill that runs AEO (Answer Engine Optimization, SEO for AI search engines) optimisation, we have series of claude skills that manage data in our CRM, another one that drafts emails or researches the status of the deal.
More and more of our usages of AI are verifiable such as alert resolution or AEO optimisation. This means that they can already be done with very little human input and as the models / AI agent harnesses become even better, there is a path towards having these workflows automated end to end. As an AI native company, it is one of our building principles that we as a company have to get better as AI ecosystem gets better, and building for verifiability means we build for AI agent optimisation, directly benefiting from latest AI ecosystem advancements.
Build with Us
We are just getting started, there is so much more we want to build across all our core business functions. If you are any sort of builder (engineer, GTM, design), join us, you will get to build the most interesting product & tooling and advance the accounting industry forward in leaps.