Advanced Responses: Custom Prompts and Generative Answers
Control how your agent generates responses using custom prompts, knowledge sources, and the generative answers node.
Two Ways to Make Your Agent Smarter
Imagine two kinds of restaurant staff. A waiter with a script reads your order options from a menu card — they say exactly what the card says, in the tone the manager wrote. That is a custom prompt: you write the instructions, the AI follows them.
A sommelier, on the other hand, knows the entire wine cellar. Ask them anything and they will search their knowledge, find the best match, and explain it in their own words. That is generative answers: the AI searches your knowledge sources and composes a response on the fly.
Most production agents use both. Custom prompts for structured, predictable responses. Generative answers for open-ended questions where the user could ask anything.
Custom Prompts: Three Levels
Custom prompts can be configured at three levels, each with different scope and override behaviour:
| Level | Where configured | Scope | Override behaviour |
|---|---|---|---|
| Agent-level | Agent settings, Instructions field | Applies to all conversations as a system prompt | Always active; provides baseline persona and rules |
| Topic-level | Inside a topic using a Prompt node | Applies only when that topic is active | Adds to or overrides agent-level instructions for this topic |
| Flow-level | Inside a Power Automate flow using AI Builder or HTTP to an LLM | Applies only during that flow execution | Completely independent; the agent does not see this prompt |
The agent-level prompt defines persona, tone, and guardrails. Topic-level prompts refine behaviour for specific scenarios. Flow-level prompts are for backend AI operations the user never sees.
Anatomy of a Good Custom Prompt
A well-structured custom prompt follows this pattern:
You are [persona] for [company].
## Rules
- Always respond in [language/tone]
- Never disclose [sensitive information]
- If unsure, say [fallback message]
## Context
The customer's name is {Topic.CustomerName}.
Their account type is {Global.AccountType}.
## Task
[Specific instruction for this response]
## Format
Respond in [format — bullet points, paragraph, structured list].
Keep responses under [length] sentences.Variable placeholders like {Topic.CustomerName} are replaced at runtime with actual values.
Knowledge Sources
Knowledge sources ground the agent’s generative answers in your organisation’s actual content. Five source types are available.
| Feature | SharePoint | Websites | Uploaded Files | Dataverse | Azure AI Search |
|---|---|---|---|---|---|
| What it indexes | Pages and documents in SharePoint sites | Public or internal web pages (crawled by URL) | PDF, Word, Excel, text files uploaded directly | Rows in Dataverse tables | Content from an existing Azure AI Search index |
| Auth required | Yes — user or agent must have SharePoint access | No for public sites; yes for authenticated pages | No — files are uploaded to the agent | Yes — Dataverse connection reference | Yes — API key or managed identity to the search service |
| Update frequency | Near real-time — picks up SharePoint changes automatically | Periodic crawl — you set the schedule | Manual — you re-upload when content changes | Real-time — queries Dataverse at runtime | Depends on your indexer schedule in Azure |
| Best for | Policy documents, HR handbooks, internal wikis | Product documentation, public FAQ pages | Static reference material that rarely changes | Structured business data — products, cases, accounts | Large-scale enterprise search across multiple data sources |
| Limitation | Respects SharePoint permissions — agent sees only what the user can see | Cannot crawl pages behind login without custom auth | No automatic updates — stale content risk | Works best with text-heavy columns, not binary data | Requires Azure subscription and AI Search resource |
The Generative Answers Node
The Generative answers node is a special node type in the topic canvas. When the conversation reaches this node, the AI:
- Takes the user’s latest message (and optionally the conversation history)
- Searches the configured knowledge sources for relevant content
- Composes a grounded answer based on retrieved content
- Returns the answer with optional citations
You configure the node by selecting which knowledge sources to search and setting content moderation level, citation style, fallback message, and search scope.
Exam tip: Generative answers vs custom prompts
This is a critical distinction for the exam:
Generative answers = the AI searches your knowledge sources and composes a response. You control what data the AI can access. The AI decides how to use that data.
Custom prompts = you write the instructions and the AI follows them. You control how the AI responds. The AI fills in the blanks using context you provide.
When the exam describes a scenario where the user “could ask anything about company policy” — generative answers. When the scenario says “the agent must always respond in a specific format with specific tone” — custom prompt.
Most production agents use both: agent-level custom prompt for persona and rules, plus generative answers nodes for open-ended knowledge questions.
🏦 Kai Configures Knowledge for Pacific Mutual
Kai is setting up the insurance agent at Pacific Mutual (15,000 employees). The agent needs to answer questions about policy coverage, claims procedures, and regulatory requirements.
Step 1: Agent-level custom prompt — Kai writes the system instructions:
You are the Pacific Mutual Insurance Assistant. You help customers
and internal staff understand policy coverage, claims procedures,
and regulatory requirements.
## Rules
- Always cite the specific policy section when answering coverage questions
- Never provide legal advice — direct users to the Legal team for disputes
- For claims status, always redirect to the claims tracking topic
- Respond in a professional but friendly toneStep 2: SharePoint knowledge source — Kai connects the agent to the Policy Documents SharePoint site. This contains 200+ PDF policy documents, the employee handbook, and the claims procedure guide. The agent respects SharePoint permissions — internal staff see all documents; customers only see public-facing policies.
Step 3: Dataverse knowledge source — Kai adds the Products table in Dataverse as a knowledge source. This gives the agent real-time access to current plan details, premiums, and coverage limits — data that changes quarterly and would be stale in uploaded files.
Step 4: Website knowledge source — Kai points the agent at Pacific Mutual’s public FAQ pages so common questions surface with links to source pages.
Step 5: Topic-level prompt for claims — The claims topic adds: “Express empathy first. Acknowledge the situation before procedural information. Always ask for the claim reference number.”
The result: a layered architecture where the agent draws from multiple sources and adjusts behaviour based on context.
Key Terms
Knowledge Check
Kai wants Pacific Mutual's agent to answer open-ended questions about policy coverage using the 200+ policy documents stored in SharePoint. Which approach should he use?
Priya at AgentForge is building an ISV agent that must always respond in formal British English, never mention competitors, and include a disclaimer on financial topics. Where should she configure these rules?
An agent has a generative answers node that cannot find relevant content for a user's question. What happens?
🎬 Video coming soon
Advanced Responses: Custom Prompts and Generative Answers
Next up: API Calls, HTTP Requests and Adaptive Cards — connect your agent to external APIs, parse JSON responses, and present rich interactive cards to users.