Agents in Foundry: Create & Test | Guided by A Guide to Cloud

What is an AI agent?

Simple explanation

An AI agent is a chatbot with superpowers — it can actually DO things, not just talk.

A regular chat model answers questions. An agent answers questions AND takes action: it can search databases, send emails, book meetings, analyse data, and call external services — all automatically.

Think of the difference between asking someone “What’s the weather?” (chat) vs asking “Check the weather, and if it’s going to rain, cancel my outdoor meeting and send an email to the team” (agent).

Chat model vs agent

Chat model vs AI agent
Feature	Chat Model	Agent
What it does	Responds to prompts with text	Plans and executes tasks using tools
Uses tools?	No — text in, text out only	Yes — can call APIs, search, query databases
Autonomous?	No — responds only when prompted	Yes — can plan multi-step workflows
Memory	Only within the message history you send	Can persist memory across conversations
Example	'What's the capital of France?' → 'Paris'	'Book a meeting with the marketing team next Tuesday at 2pm' → searches calendars, sends invites

Agent components

An agent in Foundry has four key components:

Component	What It Does	Example
Model	The LLM brain that reasons and plans	GPT-4o
Instructions	The agent’s system prompt — its role, rules, personality	”You are DataFlow’s IT helpdesk agent. Always check the KB before escalating.”
Tools	External capabilities the agent can use	Search knowledge base, create ticket, send email
Knowledge	Data sources the agent can search (Foundry IQ)	Company FAQ, product docs, policy manuals

Creating an agent in the Foundry portal

Step 1: Navigate to Build → Agents

In the Foundry portal, go to the Build section and select Agents.

Step 2: Create a new agent

Setting	What to Configure
Name	A descriptive name (e.g., “IT Helpdesk Agent”)
Model	Select a deployed model (GPT-4o recommended for agents)
Instructions	Write the system prompt — role, rules, personality, boundaries

Step 3: Add tools

Tools give your agent capabilities beyond text generation:

Tool Type	What It Does	Example
Code interpreter	Runs Python code for calculations, data analysis, charts	”Calculate the average ticket resolution time from this CSV”
File search / Foundry IQ	Searches uploaded documents for answers	”Find the policy on remote work in the employee handbook”
Function calling	Calls custom APIs or functions you define	”Look up order #12345 in our system”
Azure AI Search	Queries an Azure Search index	”Find all knowledge base articles about password resets”

Step 4: Test in the portal

The Foundry portal includes a test chat where you can:

Send messages to your agent
Watch it decide which tools to use
See tool inputs and outputs
Verify the final response

DataFlow Corp scenario: DataFlow creates an IT helpdesk agent:

Model: GPT-4o
Instructions: “You are DataFlow’s IT helpdesk assistant. Always search the knowledge base before suggesting solutions. If you can’t find an answer, create a support ticket. Be professional and empathetic.”
Tools: Knowledge base search, ticket creation API, email notification
Test: “My laptop won’t connect to WiFi” → Agent searches KB → finds WiFi troubleshooting guide → provides step-by-step solution → asks if it helped → if not, creates a ticket

How agents decide which tool to use

When an agent receives a user message, the LLM reasons about what tools to use:

User says: “What’s our refund policy for orders over $500?”
Agent thinks: “I need to search the knowledge base for the refund policy”
Agent calls: File search tool with query “refund policy orders over 500”
Tool returns: Relevant policy document excerpts
Agent generates: A response grounded in the actual policy

The LLM decides tool selection — you don’t need to program “if user asks about refunds, search KB.” The agent figures this out from its instructions and the available tools.

🎬 Video walkthrough

🎬 Video coming soon

Agents in Foundry — AI-901 Module 15

~14 min

Flashcards

Question

What are the four key components of an agent in Microsoft Foundry?

Click or press Enter to reveal answer

Answer

Model (the LLM brain), Instructions (system prompt with role and rules), Tools (external capabilities like APIs and search), and Knowledge (data sources the agent can search).

Click to flip back

Question

What is function calling in an AI agent?

Click or press Enter to reveal answer

Answer

The ability for an agent to call custom APIs or functions you define. The LLM decides when to call a function based on the user's request, sends the appropriate parameters, and uses the result in its response.

Click to flip back

Question

How does an AI agent decide which tool to use?

Click or press Enter to reveal answer

Answer

The LLM reasons about the user's request and its available tools, then selects the most appropriate tool to call. You don't need to program explicit rules — the model figures it out from its instructions and the tool descriptions.

Click to flip back

Question

What is the Responses API?

Click or press Enter to reveal answer

Answer

The current API for building agents in Microsoft Foundry (successor to the Assistants API). It supports conversations, tool calling, memory, and multi-agent workflows.

Click to flip back

Knowledge Check

MediSpark creates an agent to help patients schedule appointments. The agent needs to check doctor availability, find open slots, and book the appointment. What component makes this possible?

Knowledge Check

DataFlow Corp's IT helpdesk agent receives the question: 'How do I reset my VPN password?' The agent searches the knowledge base, finds the relevant article, and provides step-by-step instructions. Which agent capability was most critical here?

Next up: Building an Agent Client App — connecting your Python code to an agent.