---
title: "The eve Agent Framework Is Next.js for AI Agents: What I Learned Building FinanceBench on It"
date: "2026-10-25"
excerpt: "The eve framework from Vercel is for building AI agents in TypeScript. I used a real project to learn it: FinanceBench, 150 questions about real company reports, with real gold answers and real scores. The framework's conventions do the setup for you. Thus I used my time on the agent, not on the setup."
template: "technical"
category: "AI Engineering"
---
**What is eve?** It is a framework from Vercel for building AI agents in TypeScript ([eve.dev](https://eve.dev)). The name is always written in lowercase. In this post, "eve" always means this framework.

134 to 137 correct answers from 150 questions. That is 89.3% to 91.3%, depending on the run.

An agent built with eve got these scores on FinanceBench. The model was DeepSeek V4.1 Flash. The best run was 137/150. The latest run was 134/150.

[FinanceBench](https://github.com/patronus-ai/financebench) is a public benchmark from Patronus AI. It has 150 open-source questions about real company reports: 10-K, 10-Q, 8-K and earnings reports. Analysts wrote a gold answer for each question. Two example questions:

- "Does Corning have positive working capital based on FY2022 data?"
- "What is the FY2017 return on assets for Coca Cola?"

The [FinanceBench paper](https://arxiv.org/abs/2311.11944) also gives a score for GPT-4-Turbo. In that test, GPT-4-Turbo got the full report in its context. Its score was 79%. That score uses the paper's grading and the original gold answers. Our later scores use a stricter number check and 5 corrected gold answers. Thus this is not an exact comparison. Our first baseline used the original gold answers. It got 121/150 (80.7%).

These scores are real. The benchmark is real. That was my aim.

---

## Why I used a real project

I want to learn the eve framework. Then I want to teach it to you.

A todo app does not teach much. It does not fail in a useful way.

A benchmark is different. Each question has a correct answer. The benchmark tells you when your agent is wrong. Also, this benchmark is like our work. The agent reads a long document. It finds the numbers. It calculates. It gives an answer and shows the evidence.

---

## The eve framework is like Next.js for agents

The eve framework is like Next.js. Convention over ceremony.

In Next.js, a file in `app/` becomes a route. In eve, a file in `agent/` becomes a part of the agent:

| File or folder | What it becomes |
|---|---|
| `agent/instructions.md` | The system prompt |
| `agent/agent.ts` | The model and the runtime settings |
| `agent/tools/*.ts` | Tools that the model can call |
| `agent/skills/*.md` | Procedures that the model loads only when it needs them |
| `agent/sandbox/` | The workspace and the isolated machine for the agent |
| `evals/*.eval.ts` | Tests for the agent |

You do not connect these parts. You put a file in the correct folder. The framework finds the file.

For most other tasks, you use one command. The framework has its own CLI. In this project, I used `eve link` and `eve eval`. The CLI also has commands such as `eve add` and `eve deploy`. Thus the instructions for a task are short.

The best practices are part of eve. You see what to do, rather than how to do it.

In the first hour of work, I created the agent and ran it with an eval. It is just there. Everything is clear on what to do.

---

## What I did not have to build

### Model change

The model configuration of this agent has 7 lines:

```ts

// FB_MODEL picks the answering model for model comparisons (Story 8); same reasoning for all.
defineAgent({
  model: process.env.FB_MODEL || "deepseek/deepseek-v4.1-flash-1017",
  reasoning: "high",
});
```

To compare models, I changed one environment variable. I ran the same evals on five models:

- MiniMax M3
- Claude Haiku 5.5
- GPT-6 Luna
- GPT-6 Luna Fast
- DeepSeek V4.1 Flash

I did not write new client code. I did not add a new SDK.

### API keys

`eve link` connects the project to Vercel. Then the agent calls the models through the Vercel AI Gateway.

This repo has no OpenAI key, no Anthropic key and no DeepSeek key. It has only one secret: a Vercel token in `.env.local`. The link step wrote this token.

### The sandbox

The agent must read 84 reports. It must not use the internet. The sandbox file has 10 lines:

```ts

const environment = DockerSandbox.environment();

// No network: answers must come from the reports in /workspace/reports.
defineSandbox(() => environment.open({ networkPolicy: "deny-all" }));
```

The reports are in `agent/sandbox/workspace/reports/`. For each session, the framework copies the reports into a new Docker container. The agent reads the reports with `grep` and `sed` in `bash`.

Each question gets a clean workspace, without hard labour.

To remove a built-in tool, you write one file. The file has one line of logic:

```ts

disableTool();
```

### Evals and traces

Without eve, this is the part I would spend the most time on.

There is one eval for each question. Each eval does three things:

1. It sends the question to the agent.
2. It makes sure that the reply has an `ANSWER:` line.
3. It tells a judge model to compare the answer with the gold answer.

```ts
const turn = await t.send(questionMessage(q));
t.succeeded();
t.check(turn.message, includes("ANSWER:"));
t.judge({ state: judgeState(q, turn.message ?? ""), questions: { correctness: JUDGE_QUESTION } });
```

`eve eval` runs all 150 evals. It runs 8 evals at the same time. A full run takes approximately 10 minutes.

Each run writes its results to `.eve/evals/<timestamp>/`. The results include a full trace for each question. The trace shows each model step, each tool call, each command and the cost. I did not configure this.

When an answer was wrong, I opened its trace. The trace showed what the agent searched for and what it read.

This is the most important thing that eve gave me. I worked on the logic and on improvements, not on the configuration of the application.

---

## The scores

This table shows the score history. Read the notes below the table.

| Version | Model | Judge | Score | Model cost |
|---|---|---|---|---|
| First baseline | MiniMax M3 | v1 | 121/150 (80.7%) | $1.05 |
| Better instructions + ratio formulas | MiniMax M3 | v2 | 132/150 (88.0%) | $1.09 |
| Formulas moved into the instructions | MiniMax M3 | v3 | 137/150 (91.3%) | $1.09 |
| Different model | DeepSeek V4.1 Flash | v3 | 137/150 (91.3%) | $0.16 |
| Three more instruction rules | DeepSeek V4.1 Flash | v3 | 134/150 (89.3%) | $1.11 |

The agent did not make all of this increase. These are the other causes:

- **The judge changed.** Judge v3 uses code to compare numbers. The judge model could not always see that 1.42% and 0.01 are the same value with different rounding.
- **Some gold answers changed.** An AI coding agent examined each failed question against its report on my instruction. It found 5 gold answers that were wrong:
  - one arithmetic error,
  - one fact that was out of date,
  - one sign that was reversed,
  - one value with incorrect rounding,
  - one incorrect conclusion.

  I did not examine these corrections myself. The agent wrote the report file and line numbers for each correction. The corrections are in a separate file (`evals/data/gold-corrections.json`), with that evidence. You can examine each one against the report. The original dataset did not change.
- **One run does not prove a result.** The judge gave the same answer (36.5%) a "correct" label in one run and an "incorrect" label in the next run. The last row is 3 questions lower than the row above it. Most of this difference is from this judge noise. Do not accept a change of less than approximately 3 questions until you see it in more than one run.
- **The cost changed, but the code did not.** The last run used the same quantity of tokens as the run above it. But it cost 7 times more. Possibly, the gateway sent the requests to a more expensive provider.

The DeepSeek rows and the MiniMax row for v3 use the same judge and the same corrected gold answers. Thus you can compare them directly. In the best runs, the score is the same. In that comparison, DeepSeek cost approximately one seventh.

---

## Limits

The eve framework is in preview. Its documents say that the APIs can change before general availability.

The Docker sandbox runs on your local computer. To deploy this agent on Vercel, you must change to the Vercel Sandbox. I did not do this yet.

`eve eval` has no option to repeat a run. To measure the noise, run `eve eval` more than one time. Then compare the runs yourself.

---

## What I want you to do

Use eve for your next new project. For example: lease extraction, RegTech, or a different project where an agent reads documents and must give correct answers.

The parts of this repo are the same as the parts of our work:

| FinanceBench | Our work |
|---|---|
| 84 reports in the sandbox workspace | The leases or regulations of the client |
| 150 questions with gold answers | Questions with answers that our experts verified |
| One eval for each question | One eval for each question |
| `FB_MODEL` | Compare models before you select one |
| A trace for each answer | The evidence when we must explain an answer |

Write the evals first. Then improve the agent. Monitor the score after each change.

The first commit of this project was on 24 October at 12:42. The last code commit was on 25 October at 19:37. I did other work between these commits. The code itself took less time.

The repo is at [github.com/satish860/financebench-eve](https://github.com/satish860/financebench-eve). Clone it. Run `eve eval`. Then read the traces.

---

## Frequently asked questions

**What is the eve framework?**

The eve framework ([eve.dev](https://eve.dev)) is from Vercel. You use it to build AI agents in TypeScript. You write an agent as files: instructions in `agent/instructions.md`, model settings in `agent/agent.ts`, tools in `agent/tools/`, skills in `agent/skills/`, the sandbox in `agent/sandbox/`, and tests in `evals/`. The framework finds these files by convention, like Next.js finds routes, and gives a CLI, a sandbox, evals and traces without extra setup.

**What score does an eve agent get on FinanceBench?**

An eve agent with DeepSeek V4.1 Flash answered 134 to 137 of the 150 open-source FinanceBench questions correctly (89.3% to 91.3%), depending on the run. These scores use a judge that compares numbers in code and 5 corrected gold answers. The first baseline, with MiniMax M3 and the original gold answers, scored 121/150 (80.7%). The FinanceBench paper reports 79% for GPT-4-Turbo with the full report in context.

**How do you switch models in an eve agent?**

Change the `model` field in `agent/agent.ts` to another Vercel AI Gateway model id. After `eve link` connects the project to Vercel, the agent calls models through the AI Gateway, so the repo needs no OpenAI, Anthropic or DeepSeek API key. The FinanceBench agent reads the model id from an environment variable, so one variable change runs the same evals on a different model.

**How do evals and traces work in eve?**

Each file in `evals/` that ends in `.eval.ts` defines evals with `defineEval`. An eval sends a message to the agent, checks the reply, and can ask a judge model to grade it with `t.judge`. The command `eve eval` runs the evals in parallel and writes results and a full trace for each eval to `.eve/evals/<timestamp>/`, with every model step, tool call, command and cost.

**Are FinanceBench gold answers always correct?**

No. In this project, an AI coding agent checked each failed question against its report and found 5 gold answers that contradict the report: an arithmetic error, an out-of-date fact, a reversed sign, a rounding error and a wrong conclusion. The corrections, with report line evidence, are in [`evals/data/gold-corrections.json`](https://github.com/satish860/financebench-eve/blob/master/evals/data/gold-corrections.json). No person has checked them yet.

**Can you deploy an eve agent that uses a Docker sandbox to Vercel?**

Not as it is. The Docker sandbox runs on a local computer. To deploy on Vercel, the agent must use the Vercel Sandbox instead. The eve framework is also in preview, and its documents say that the APIs can change before general availability.

---

*DeltaXY builds document intelligence for regulated industries — aviation leasing, financial compliance, legal tech. 10,000+ documents processed in production, 95% extraction accuracy. If you're wrestling with an AI document project and need someone who's actually shipped in production — I do consulting.*

**[deltaxy.ai](https://deltaxy.ai)** · **[satish@deltaxy.ai](mailto:satish@deltaxy.ai)**