Learn what to prepare — and what to decide — before you switch on an AI Agent, so it launches as a reliable member of your support team instead of a liability.
Start by treating it like a new hire
The single most useful mindset: your AI Agent is a new colleague, not a magic button.
A great new support hire needs three things before their first shift — knowledge about your business, clear boundaries about what they're allowed to do, and someone checking their early work. Your AI Agent is exactly the same. Give it those three things and it will handle a large share of your everyday questions well. Skip them and it will confidently give wrong answers, because that's what an under-prepared colleague does too.
This article walks through everything worth deciding before you turn the agent on. You don't have to perfect all of it on day one — but reading through it now will save you from the most common (and most public) mistakes.
1. Knowledge — what your agent actually knows
Your AI Agent can only answer well when it has something reliable to answer from. It doesn't "just know" your shipping times, your return window, or which products you sell — you have to give it that knowledge.
In Herodesk you feed the agent through knowledge sources. The main ones are:
- 📚 Help Center articles & FAQs — Your existing help articles and FAQs are the strongest foundation. If a customer would find the answer on your help site, the agent should be able to find it too. If you haven't written help articles yet, this is the highest-value work you can do before launch.
- 🌐 Your website — The agent can read and learn from your public website pages (about pages, shipping policy, terms, product info).
- 🛒 Product feed — If you run a webshop, connecting your product feed lets the agent answer questions about specific products, prices and availability.
- 💬 Past conversations — Previously answered tickets teach the agent how your team has handled real questions.
Think like a new colleague: Ask yourself — "If I hired someone on Monday, what would I need to hand them so they could answer customers by Friday?" Procedures for claims, returns, refunds, discount codes, delivery, and product recommendations are usually the first things to document. Anything that lives only in your team's heads is invisible to the AI.
What good preparation looks like:
- The most common 20–30 questions your customers ask are each covered by an article, FAQ, or clear page.
- Policies (returns, shipping, warranty) are written down somewhere the agent can read.
- Product information is connected if you sell products.
2. Data discipline — is your information actually correct?
Here's an uncomfortable truth: an AI Agent will repeat your mistakes faster and more confidently than any human ever could.
If your website still lists last year's shipping rates, mentions a return guide you've since changed, or references a product you discontinued, the agent will happily tell customers all of it — over and over, at scale.
Before you launch, audit your knowledge for:
- ❌ Outdated shipping rates or delivery times
- ❌ Old return or warranty policies that no longer apply
- ❌ Discontinued products still described as available
- ❌ Contradictions — where two pages give different answers to the same question
Rule of thumb: if two of your own articles disagree, the agent has no way to know which one is right. Cleaning up contradictions before launch does more for answer quality than almost anything else.
3. Role in the team — what should it handle, and what shouldn't it?
You wouldn't let a brand-new colleague issue refunds or handle a legal complaint on their first day. The same judgement applies here. Decide up front which kinds of questions the AI should own, and which should always go to a human.
Herodesk organises this around Topics. A topic is simply a category of question — "Delivery status", "Order cancellation", "Product questions", "Damaged in transit" — and each topic can be handled differently:
- Let the AI handle it fully (for safe, well-documented topics like delivery status or opening hours).
- Let the AI draft a reply for a human to review (for topics where you want a second pair of eyes).
- Always send it to a human (for sensitive topics — complaints, legal, anything involving money you're not ready to automate).
You don't have to map out every topic before launch — Herodesk will even discover the topics your customers actually write about for you. But you should decide your red lines: the handful of subjects the AI must never answer alone.
📖 We cover topics in depth in the "Getting started" series — each part walks you through one topic at a time.
4. How much control to give it — deployment modes
Separately from what the agent handles, you choose how much freedom it has when it does. On the Go live page you set this per channel (live chat, email, Messenger, Instagram, WhatsApp) using four modes. Think of them as a dial you can turn up gradually as trust grows:
| Mode | What happens | Best for |
|---|---|---|
| 🔴 Off | The agent does nothing on this channel. | Channels you're not ready to automate. |
| 📝 Review | The agent writes a suggested reply, but a human must send it. Nothing reaches the customer automatically. | Starting out — see exactly what the AI would say, with zero risk. |
| 🤝 Assist | The agent replies to customers on its own, but any action (like cancelling an order) is only proposed — a human approves it before it happens. | Building trust while keeping a hand on the wheel for real actions. |
| 🚀 Auto | The agent replies and performs allowed actions on its own. | Channels you've tested and trust. |
Our strong recommendation: start in Review mode. It lets you watch the agent answer real customer questions on your real channels — without a single message being sent — so you can judge quality before anything is public. It's also free to run in Review mode, so there's no reason not to.
💡 "Auto" vs "Autopilot": The channel setting on the Go live page is called Auto. Separately, once a specific topic has proven itself, you can promote it to Autopilot on the Topics board — which lets the agent handle that topic on its own even on channels you've set more cautiously. Two dials, working together: the channel sets the ceiling, the topic fine-tunes it.
5. Tone of voice — give it a personality
Your AI Agent represents your brand in every reply, so it should sound like your brand. Herodesk gives you:
- Ready-made personality presets for a quick start.
- A free-text description where you can describe your brand voice in your own words — "warm and playful", "concise and professional", "friendly but never over-promising".
- A Custom Prompt for specific behaviours — for example, always greeting customers by name, always framing bad news positively with a clear next step, or never quoting delivery dates it isn't certain about.
A few things worth deciding before launch:
- How formal or casual should it be?
- Should it use the customer's name? Emojis?
- Are there phrases or claims it must never make (e.g. promising refunds, guaranteeing delivery dates)?
You can refine all of this later, but setting a clear voice up front makes the agent feel like your colleague from the first reply.
6. Plan the handover — how it steps aside gracefully
A good colleague knows when a question is beyond them and passes it on cleanly. Your AI Agent does this through handover — when it can't help, it hands the conversation to your team instead of guessing.
Handover happens automatically when:
- The customer asks for a human.
- The agent can't answer confidently after a couple of tries.
- The topic is one you've marked as human-only.
Before launch, think about:
- Who picks up handed-over conversations? Make sure they land in the right inbox/team.
- What should the agent say as it hands over? A short, reassuring message ("Let me pass you to a colleague who can help with that") keeps the experience smooth.
- Your opening hours (for live chat) — the agent can be set to only reply when your team is available to back it up.
7. Transparency — tell customers it's AI
It's tempting to hide the fact that an AI is answering. Don't. Customer satisfaction is consistently higher when you're transparent.
When customers know they're talking to an assistant — for example, one clearly named "[Your brand] AI Assistant" — they:
- Have clearer expectations of the kind of answers they'll get.
- Are more forgiving of a quick clarifying question.
- Trust you more, because you're being upfront.
Decide on a name and an introduction for your agent that makes its role obvious, friendly, and on-brand.
8. Test before you go live
Never let your first real customer be the test. Herodesk gives you two safe ways to check the agent's quality before it sends anything:
- 🧪 Playground — A private, internal sandbox. You (and your team) chat with the agent as if you were a customer, try your trickiest questions, and see exactly how it responds. Nothing here reaches real customers.
- 📝 Review mode — As described above, run the agent on your real channels in Review mode. It drafts replies to genuine customer questions that you read (and choose whether to send). This is the most realistic test there is, and it costs nothing.
When you spot a weak answer, don't just fix the wording — ask why:
- Was the information missing from your knowledge sources? → Add or update an article.
- Was the information wrong or outdated? → Correct the source.
- Was the tone off? → Adjust the personality or custom prompt.
- Should the AI not have answered at all? → Move that topic to human-only.
Herodesk shows you which sources the agent used for each reply, so you can trace a bad answer back to its cause quickly.
9. Plan to keep improving
Launch isn't the finish line — it's the start of coaching. Just like a real colleague, your agent gets better with feedback. Herodesk lets you fine-tune the agent over time: when you edit one of its suggested replies before sending, that correction becomes a lesson the agent can learn from.
You don't need to set this up before launch — just know that "improving over time" is part of the model, and we cover it fully in the "Fine tuning your AI Agent" article.
Your pre-launch checklist
Before you switch the agent on, make sure you can tick these off:
- My most common customer questions are covered by help articles, FAQs, or clear pages.
- My key policies (returns, shipping, warranty) are written down and up to date.
- I've audited for outdated info and contradictions.
- I've connected the right knowledge sources (help center, website, product feed).
- I've decided which topics are off-limits to the AI.
- I've chosen a starting deployment mode (we recommend Review).
- I've set a personality / tone of voice that fits my brand.
- I've planned the handover — who catches conversations the AI passes on.
- I've decided how I'll be transparent that it's an AI.
- I've tested in the Playground and/or Review mode before going live.
Need help?
If you have questions about setting up your AI Agent — or anything else in Herodesk — reach out to us at support@herodesk.com. We're happy to help you get your digital colleague ready for their first shift. 🙌