Information Extraction: From Chaos to Structure

What is information extraction?

Extraction across modalities

Information extraction across four modalities

Feature	What AI Extracts	Example
📄 From text/documents	Specific fields from forms, invoices, contracts, reports	Invoice number, supplier name, line items, total amount
🖼️ From images	Text, objects, labels, and metadata from photos	Product label info, building permit numbers, medical chart readings
🎙️ From audio	Spoken content, speaker identity, key phrases, topics	Meeting action items, interview highlights, customer complaints
🎬 From video	Visual content, spoken words, on-screen text, scenes	Presentation slide text, training video topics, security footage events

Document extraction

The most common extraction scenario. AI reads structured and semi-structured documents and extracts specific fields.

Document Type	Fields Extracted
Invoices	Invoice number, vendor, date, line items, total, tax
Receipts	Store name, items, prices, total, date
ID documents	Name, date of birth, document number, nationality
Health records	Patient name, diagnosis, medications, dates
Contracts	Parties, dates, terms, obligations, amounts

GreenLeaf scenario: GreenLeaf receives hundreds of supplier invoices per month in different formats — some printed, some scanned, some handwritten. Content Understanding reads each one and extracts the vendor name, amounts, and payment terms into their accounting system.

How extraction differs from text analysis

Text analysis vs information extraction

Feature	Text Analysis	Information Extraction
Goal	Understand meaning and sentiment	Pull out specific data fields and values
Input	Usually clean text	Documents, images, audio, video (messy/varied)
Output	Sentiment scores, keywords, entities, summaries	Structured data: { field: value } pairs
Example	'This review is 85% positive'	'Invoice #4521, Total: $3,400, Due: 15 May 2026'
Azure service	Azure AI Language	Azure Content Understanding

Azure Content Understanding

Azure Content Understanding is the Azure service for multimodal information extraction. It’s part of Foundry Tools and can process:

• Documents and forms (PDF, images of forms)
• Images (photos, screenshots)
• Audio (recordings, calls)
• Video (presentations, training content)

You’ll work hands-on with Content Understanding in Domain 2 (Modules 24-27).

ℹ️ How Content Understanding works under the hood

Content Understanding combines multiple AI capabilities:

1. OCR — reads text from the document/image
2. Layout analysis — understands tables, headers, paragraphs, and document structure
3. Field extraction — maps specific regions to named fields
4. Validation — checks extracted data against expected formats (dates, numbers, etc.)

For audio and video, it adds: 5. Speech recognition — transcribes spoken content 6. Scene detection — identifies key moments in video 7. Slide extraction — captures on-screen text and slides

This multimodal approach means you can build one extraction pipeline that handles documents, images, audio, AND video.

🎬 Video walkthrough

🎬 Video coming soon

Information Extraction — AI-901 Module 11

Information Extraction — AI-901 Module 11

~12 min

Flashcards

Question

What is the difference between text analysis and information extraction?

Click or press Enter to reveal answer

Answer

Text analysis understands meaning (sentiment, keywords, entities). Information extraction pulls out specific data fields and values from unstructured content (documents, images, audio, video). Analysis = understanding. Extraction = structured output.

Click to flip back

Question

What is Azure Content Understanding?

Click or press Enter to reveal answer

Answer

A Foundry Tools service for multimodal information extraction. It can process documents, forms, images, audio, and video — extracting structured data fields from unstructured content.

Click to flip back

Question

What four modalities can information extraction work with?

Click or press Enter to reveal answer

Answer

Text/documents (invoices, forms), images (photos, labels), audio (recordings, calls), and video (presentations, security footage).

Click to flip back

Question

How does Content Understanding process a scanned invoice?

Click or press Enter to reveal answer

Answer

1) OCR reads the text, 2) Layout analysis understands tables and structure, 3) Field extraction maps regions to named fields (invoice number, total), 4) Validation checks data formats.

Click to flip back

Knowledge Check

Knowledge Check

MediSpark receives patient intake forms in multiple formats: some typed PDFs, some scanned handwritten forms, some photographed with phones. They need to extract patient name, DOB, and insurance number from all of them. Which Azure service is best suited?

AAzure AI Language — for text analysis of the form contentBAzure AI Vision — for image classification of the formsCAzure Content Understanding — for multimodal extraction of structured fields from documents and imagesDAzure AI Speech — to read the forms aloud

Check Answer

Knowledge Check

DataFlow Corp records all customer support calls. They want to extract: the customer's account number (spoken), the issue category, and the resolution provided. Which modality of information extraction is this?

ADocument extraction — the call is like a documentBImage extraction — the audio contains visual informationCAudio extraction — extracting structured information from spoken contentDVideo extraction — calls include visual elements

Check Answer

---

🎉 You’ve completed Domain 1! You now understand AI concepts, responsible AI, model types, deployment, and all six workload categories. Domain 2 takes you hands-on — building real AI solutions in Microsoft Foundry.

Next up: Prompting Fundamentals — crafting effective system and user prompts for generative AI models.