Track AI agent goals and fine-tune replies with MCP

Managers can ask Claude or ChatGPT to set up goals for an AI agent, report on them, and correct the agent's replies so it answers better next time.

Written by Anders Eiler
Last updated 2026-11-18
On this page

With the Herodesk MCP server, a manager can work with AI agent goals and fine-tuning from their AI assistant, the same way as on the Goal tracking and Fine-tuning pages in Herodesk. You need a connected assistant, see What is the Herodesk MCP server?, and the manager or admin role. Writing needs write access.

Goals

A goal says what counts as a success after your AI agent helped a customer:

  • URL goal: the customer visits a page that matches a pattern, for example https://shop.dk/checkout/thank-you* (* is a wildcard). Each visit is worth a value you set, in EUR.
  • E-commerce goal: the customer places an order in one of your connected webshops. Each webshop can be tracked by one goal per AI agent.

A conversion only counts if it happens within the attribution window after the agent's conversation, 48 hours unless you choose otherwise (1 to 720 hours).

TaskExample prompt
See the goals"Which goals does our AI agent Sam have?"
Create a goal"Add a goal for Sam: a visit to shop.dk/thank-you counts as a conversion worth 40 EUR."
Pause or change a goal"Pause the Checkout goal." / "Count conversions up to 72 hours after the conversation."
Report"How many conversions did Sam's goals get last month, and what were they worth?"
Look at single conversions"List the orders attributed to Sam this week with their conversation."

The report covers at most two years at a time. Deleting a goal also deletes the conversions it recorded, so pause it instead if you want to keep the history.

Fine-tuning

Fine-tuning teaches the agent how it should have answered. Your assistant can:

  • list the conversations where the agent replied itself and that nobody has reviewed yet,
  • read a reply together with the knowledge the agent used for it,
  • save a better reply and the reason for the correction,
  • mark a conversation as reviewed, so it leaves the list.

Example prompts:

  • "Show me Sam's conversations from the last week that still need review."
  • "Read Sam's reply in conversation #4821 and tell me which sources it used."
  • "That reply forgot our 30-day return policy. Save a better version: …"
  • "Mark conversation #4821 as reviewed."

Important: the customer keeps the reply they received. The correction is only used to teach the agent for future replies.

Only replies the AI agent sent can be fine-tuned, not customer messages or internal notes. Your assistant only sees conversations in the inboxes you have access to, the same as in Herodesk.