1Change first
No source in the answer
6 in 10 answers
Pre-launch Opening to the first product teams
Make your AI product better every release. Not just different.
Eva ties what your users need to the code behind it. When an evaluation falls short, the result points at the code, and Eva drafts the pull request for your team to review. When you ship, Eva measures again, so you know whether the change worked and what to change next.
T2Asked after a chat: "Could you reach a person when the bot couldn't help?" · 25 users
R74 · PR #142: show "Talk to a person" from the first replyeva/handoff-always → main +6 −2 · ready for review
src/chat/ChatWindow.tsx@@ -12,4 @@
1212const messages = useMessages()
1313const failedTries = countUnhelpful(messages)
14−const showTalkToPerson = failedTries >= 3
14+const showTalkToPerson = true
1515return <Chat handoff={showTalkToPerson} />
T2 asked again after the merge14 of 25→2 of 25Met
What users said, what you changed, what it did. Connected, every release.
You want to build the best product for your users. So you ship changes every week, talk them through with your team, and ask your users what they think. But what you learn ends up everywhere: in tickets, in Slack, in a survey, in someone's head. Nothing points at anything, the same question gets asked twice, and answers come in that never turn into a change. Eva connects what you know about your product with what you learn from evaluating it, so every iteration counts.
Ticket #4521
where did the bot get this from??
Slack, #ai-feature
"customers keep asking for the source"
prompt v13
"did that help?" Nobody knows.
Survey, Q3
"it just does things" · 6 of 25 positive
Ticket #4588
it booked a flight before I said yes
Dashboard
thumbs down 14% this week
Interview notes
"I stopped trusting it without a source"
PR #142
pass the docs to the prompt
Slack, #ai-feature
"is the price it shows final?" asked 9 times
prompt v12
the version before. Or after?
Ticket #4602
answer was wrong?
Eval run
41% cite a source
Answers show where they came from
3 pieces of evidence → answer.ts → PR #142 → 41% → 78%
Users feel in control of the answer
survey + ticket → agent/tools.ts → PR #151 → 6 → 19 of 25
Hands over to a person when unsure
2 pieces of evidence → escalate.ts → no change yet → 18% guesses
Price shown is the final price
9 Slack messages → no criterion yet → to your designers
Answers stick to the facts
Eva judged 120 → summarize.ts → no change needed → met twice
Remembers what the user said earlier
diary, day 4 → chat/memory.ts → PR #158 → 52% → 96%
Asks before taking an action
ticket + survey → agent/tools.ts → PR #151 → 9 → 0 unasked
Stays short and to the point
Eva judged 120 → report/prompt.ts → PR #160 → 30% → 88%
Users say the answer helped
12 of 25 asked → app/chat → waiting → too few answers
You never start from zero again.
Eva measures the same criteria before and after every change, and keeps each one with the evidence behind it, the change it led to, and what that change did. Each release adds a round, and each round starts from everything the last one found. Two releases later, your team can still say why the search was changed and whether users noticed. That record is what Eva is. The pull requests are what it produces.
What your users needBeforeAfter the changes
- Facts match the sourceMetMet
- Answers show a sourceMet after pull request #142Not metMet
- Replies in the user's languageMetMet
- Hands over to a person when unsureNot metNot met
- Users say the answer helpedToo few answersToo few answers
- Stays short and to the pointMetMet
- Remembers what the user said earlierMet after pull request #151Not metMet
- Asks before taking an actionNext to changeToo few answersNot met
- Never shows another user's dataMetMet
Criteria met4 of 96 of 9
Eva works in your repo like a colleague: through pull requests.
Your part of the setup is the first commit. Everything after that arrives as pull requests your team can read, comment on and merge.
-
Monday
S
Add Eva to the model client
-
Monday
Add Eva's map of your AI features
-
Thursday
Give the model the context it was missing
-
A week later
Criteria met: 5 of 9 → 8 of 9
Eva never merges its own work and never pushes to your main branch.
Pick the problem you recognise. See how Eva solves it.
Example: a support chatbot that answers your customers' questions using your help pages. Eva works the same way for search, writing tools, agents and any other AI feature.
With Eva: Eva keeps every change: what was changed, why, and whether it helped your users.
- Problem
Most answers don't show a source
- Cause
The chatbot finds the right help page, but answers without reading the page
- Change
Use the help page in the answer PR #142
- Result
Answers with a source: 4 in 10 → 8 in 10
With Eva: Eva measures every criterion your team wrote down and shows which ones are furthest from their pass mark, so you know what to change first.
2Next
Makes up an answer
2 in 10 answers
3Then
Doesn't get a person
1 in 10 answers
With Eva: Eva points each result at the part of your code that produced it, so nobody has to guess.
The symptom
Answers don't show a source
answer.ts✗ Answers without reading the page
retrieve.ts✓ Finds the right page
Sources.tsx✓ Shows a link when there is one
With Eva: After every change, Eva measures all your criteria again, so you see side effects before your users do.
Before
The chatbot answers from memory
Shows where the answer came from✗
Sticks to the facts✓
Gets a person when unsure✓
After Eva's change
The chatbot reads your help page first
Shows where the answer came from✓
Sticks to the facts✓
Gets a person when unsure✓
The source is there now. Nothing else got worse.
With Eva: Eva suggests the actual change, ready for you to review.
EvaMost answers don't show a source. The chatbot finds the right help page, but answers without reading the page. I've made a small change so the chatbot reads the help page first.
PR #142
SSofie
Looks good, merging.
EvaA week later: 4 in 10 → 8 in 10 answers show their source. Nothing else got worse.
With Eva: Eva judges every answer against your criteria, and asks your users whether it helped.
How do I export my data?
Go to Settings → Export.
Did this help?👍 Yes👎 No
Customers who got help
With Eva: Write it down as a criterion once. Eva measures it again after every change.
Always show where the answer came from
Get a person when unsure
Never show one customer's details to another
✋
Not met after a changeA change broke one of your criteria
Someone changed how the chatbot searches. In the next round, two judged answers showed another customer's email. The criterion went to not met, and Eva proposed limiting the search to the customer who is asking.
The same problem, without Eva and with Eva.
A team ships an AI support assistant that answers from their docs. Users start asking where the answers come from, because the assistant never links to its source.
1You find out something is wrong
Without Eva
A ticket about an unsourced answer lands in a spreadsheet with 200 others.
With Eva
Eva checks every answer. Only 41% link to the docs page they used. The team asked for 60%.
2You find the cause
Without Eva
The team meets to guess. A developer spends a day reading code.
With Eva
The result points straight to the file that builds the answer.
3You change it
Without Eva
Someone adds a line to the prompt and hopes for the best.
With Eva
Eva writes the change as a pull request that already passes your checks. A developer reviews it and merges.
4You check that it worked
Without Eva
Nobody knows. Maybe the complaints stop. Maybe the next one arrives in two weeks.
With Eva
The next round measures the same thing again. Now 78% link to their source.
Questions teams ask us.
Who is Eva for?
Product teams with an AI feature in front of real users, who want to know whether it does what those users need, and what to change when it doesn't.
Does my product have to use AI?
Eva is built for products with an AI feature. The questions to your users work for the rest of the product too; the judging of answers applies to the parts that use a model.
How does Eva check my AI's answers?
Eva uses AI models to judge each answer against the criteria your team agreed on. Questions that only your users can answer, such as whether an answer solved their problem, are asked to your users instead.
Will Eva change my code on its own?
No. Eva proposes a change for each criterion that isn't met. Code is written only for the changes you accept, and they arrive as one pull request that a person on your team merges.
Is Eva a one-off audit?
No. An audit tells you where you stand once. Eva keeps the criteria, the evidence and the changes from every round, so the next release is measured against what you already know. The longer it runs, the better it knows your product and your users.
Does Eva only talk to developers?
No. Pull requests go to your developers. When what users say raises something no criterion covers yet, such as a topic they keep bringing up or a value that is missing, Eva puts it in front of your designers as a new topic. It does not turn that into code.
What does Eva keep from my code and my AI?
Eva keeps what it works out from your code, such as the map of your product's parts, and adds it to your own repository in a folder called .eval/ so you can read it. Your AI's answers are kept only as a fingerprint by default. You can choose to keep the text.
What does it cost, and when can we start?
Eva is in pre-launch and opening to a small group of teams first. Request access below and tell us what you want to evaluate. We'll get back to you.
17 of 19 people building AI products couldn't turn their evaluation results into changes.
That is the finding of a study led by Willem van der Maden, co-founder of Eva, published at ACM CHI 2026. Research has pointed the same way for 20 years: finding problems is easier than knowing what to change. Eva is built to close that gap.
17 of 19 practitioners gathered evaluation data but couldn't turn it into concrete improvements.
Most evaluation of AI products is a spreadsheet of opinions, or a benchmark that measures something other than what users need. Neither tells you what to change. We're building Eva so that every result can be traced to the code behind it, and every change is measured again.
Willem van der Maden
Co-founder · Researcher at the IT University of Copenhagen
Researching and publishing on evaluation in AI design since 2019.
Victor Brorson
Co-founder · Engineering
Developed the method behind Eva at the IT University of Copenhagen, and now builds the product.
Be one of the first teams to evaluate with Eva.
Eva is in pre-launch. Tell us a little about your product and what you want to know about it. We'll get back to you.