UserProof

Check whether a shipped fix worked, for founders who ship site fixes with a coding agent

Updated . By Ryan Doyle, founder of UserProof, a session replay tool for coding agents.

UserProof checks a shipped fix by comparing the same page before and after, against your site's other pages, with a 95% interval. It reports bounce, next step and conversion, and says inconclusive when traffic is thin. It is for founders who ship site fixes with Claude Code, Codex or Cursor and need to know if the change moved signups.

The one worked example: Sales.co shipped one change for phone visitors from x.com, and the share who stayed past the first screen went from 24% to 47% in six days. The counts and intervals are below. This page shows a fix check with its before and after counts and its interval, from a real site, read on October 26, 2026 (118 and 85 visits). To run the same check on your own before and after counts, use the free fix check calculator.

Jump to: ways to tell · worked example · how to use it · limits · price · FAQ

Ways to tell if a site change worked

MethodWhat it needsWhat it tells you
UserProof's fix check oursThe script tag recording, the fix marked applied, 30 human sessions on the page after it. No split of traffic.Before vs after on the same page, against other pages, with a 95% interval and a plain verdict.
An A/B test toolTwo versions built before launch, traffic split between them, and usually weeks of traffic.Which version did better, with the cause isolated, but only for changes you set up as a test.
A before/after read in your analyticsEvent tracking already in place and your own spreadsheet math.A raw difference. Nothing separates the fix from a seasonal swing or a traffic mix shift.

Worked example: Sales.co, x.com visits to the home page

The fix put a subhead and both buttons on the first phone screen. It was marked applied the moment it shipped. Six days later UserProof read the check below. This is the full Sales.co story.

x.com visits to the home pageBefore (14 days)After (6 days)
Bounced90 of 118 (76%)45 of 85 (53%)
Took a next step25 of 118 (21%)38 of 85 (45%)

The bounce drop is 23 points (95% interval 10 to 36). The difference-in-differences against the site's other pages is 39 points (95% interval 23 to 54). Conversions went from 0 to 3 (118 and 137 visits), too few to call a lift on their own. A 23-point drop from 76% needs only about 53 visits a side to call; how many visitors you need to test a change works it out for your own rate. Source: UserProof check_fix on finding 8410af88, read October 26, 2026.

How to use it

  1. Record your site. Follow the Claude Code setup; the script tag does the recording. Done looks like the agent reporting status received.
  2. Get a finding. Ask your agent "what should I work on?" (get_top_fix). Done looks like one problem with its finding id, for example a dead click or a bounce.
  3. Ship the fix, then mark it. Once the change is live, tell the agent; it calls mark_fix_applied with the finding id. Done looks like a recorded fix with its time.
  4. Set a goal. Without a conversion event, the conversion line reads "not measured"; the other two still run. Done looks like a goal on the site.
  5. Check. Ask "did the fix work?" (check_fix). Done looks like three lines: Signal, Conversion and Next step, each with a verdict and its math. If it says inconclusive, call it again after more visits.

For a fix found by the site scan, the check instead retries the defect on the live site 3 times. Pass a local dev server URL to test before you deploy.

Before a site has traffic, start with the site scan.

Limits, stated plainly

Price

PlanPriceRecorded visits a monthFix checks
Free$0 (3 sites)5,000Unlimited
Pro$29 a month50,000Unlimited
Scale$99 a month250,000Unlimited

Read from the pricing page on November 7, 2026. To size the traffic you need, try the funnel calculator.

Weighing UserProof against other tools your agent can read? Our ranking of website analytics tools for AI coding agents compares seven, with prices read November 6, 2026.

FAQ

How long after I ship a fix can UserProof check it?

As soon as the page has had 30 human sessions since the fix was marked applied. Before that the check says inconclusive and tells you how many sessions it has. With typical traffic that is a few days. It compares up to 14 days before the fix with up to 14 days after.

Does it prove the fix caused the change?

No. It compares the same page before and after, then against your site's other pages to remove site-wide drift such as a seasonal swing. It cannot separate your fix from another change that hit the same page in the same days, so mark one fix at a time.

Does it measure signups or only clicks?

Three reads come back. The signal is the problem itself, such as bounce. The next step is the share of visits that click a control, touch a form or open a second page. Conversion needs a goal you set, and it is called a lift only when the interval's lower bound is above zero. A next step is a step toward the goal, not proof of sales.

What does it cost to check a fix?

Nothing extra. Fix checks are unlimited on every plan, including Free. You pay only by recorded visits a month: Free is 5,000, Pro is $29 for 50,000 and Scale is $99 for 250,000.

Before there is a fix to check, your agent has to find the file and line to change.

Sources, read November 7, 2026