Change Feed Patterns: Materialized Views and Estimator

Change feed as an event backbone

Amara’s patterns at SensorFlow

📡 Amara at SensorFlow uses the change feed to power three critical patterns:

1. Dashboard materialized view — pre-computed “latest reading per device” for fast dashboard loads
2. Alert propagation — when a reading exceeds threshold, write to an alerts container
3. Lag monitoring — ensure processing stays within 30 seconds of real-time

Pattern 1: Materialized views

A materialized view is a pre-computed, query-optimised container that’s kept in sync via the change feed.

Source container: "readings"              Target container: "device-latest"
  partitioned by /deviceId                  partitioned by /region
  500M events/day                           10,000 documents (one per device)

Change feed processor: on each new reading,
  upsert into device-latest with the latest values

static async Task HandleChangesAsync(
    ChangeFeedProcessorContext context,
    IReadOnlyCollection<SensorReading> changes,
    CancellationToken ct)
{
    foreach (SensorReading reading in changes)
    {
        // Upsert into the materialized view container
        var latest = new DeviceLatest
        {
            id = reading.DeviceId,           // one doc per device
            Region = reading.Region,          // partition key for view
            DeviceId = reading.DeviceId,
            Temperature = reading.Temperature,
            LastUpdated = reading.Timestamp
        };

        await latestContainer.UpsertItemAsync(latest,
            new PartitionKey(latest.Region));
    }
}

Why this is powerful:

• The dashboard reads 10,000 docs from device-latest (cheap, single-partition queries by region) instead of scanning 500M docs in readings
• Different partition key on the view container enables query patterns not possible on the source

💡 Exam tip: materialized views reduce query cost

Materialized views trade write cost (change feed processing + upserts) for read cost savings (cheap reads from the pre-computed view). This is ideal when data is written once but read many times (high read-to-write ratio).

The exam may present a scenario where a complex, expensive query runs frequently. The answer is often: “Use the change feed to maintain a materialized view.”

Pattern 2: Denormalization propagation

When you embed (denormalize) data across containers, the change feed keeps copies in sync:

"products" container (source of truth)
  { id: "P1", name: "Sensor-X", price: 29.99 }

"orders" container (denormalized copy)
  { id: "O1", productName: "Sensor-X", productPrice: 29.99, ... }

When product P1's price changes:
  Change feed processor updates ALL orders referencing P1

static async Task PropagateProductChanges(
    ChangeFeedProcessorContext context,
    IReadOnlyCollection<Product> changes,
    CancellationToken ct)
{
    foreach (Product product in changes)
    {
        // Find all orders with this product and update denormalized fields
        var query = new QueryDefinition(
            "SELECT * FROM c WHERE c.productId = @pid")
            .WithParameter("@pid", product.Id);

        using FeedIterator<Order> iterator =
            ordersContainer.GetItemQueryIterator<Order>(query);

        while (iterator.HasMoreResults)
        {
            FeedResponse<Order> orders = await iterator.ReadNextAsync();
            foreach (Order order in orders)
            {
                order.ProductName = product.Name;
                order.ProductPrice = product.Price;
                await ordersContainer.UpsertItemAsync(order,
                    new PartitionKey(order.CustomerId));
            }
        }
    }
}

Pattern 3: Event-driven aggregation

Maintain running aggregates without expensive queries:

// Maintain a running average temperature per device
static async Task UpdateAggregates(
    ChangeFeedProcessorContext context,
    IReadOnlyCollection<SensorReading> changes,
    CancellationToken ct)
{
    foreach (SensorReading reading in changes)
    {
        // Read current aggregate
        var response = await aggregatesContainer.ReadItemAsync<DeviceAggregate>(
            reading.DeviceId, new PartitionKey(reading.DeviceId));

        var agg = response.Resource;
        agg.ReadingCount++;
        agg.TotalTemperature += reading.Temperature;
        agg.AverageTemperature = agg.TotalTemperature / agg.ReadingCount;
        agg.MaxTemperature = Math.Max(agg.MaxTemperature, reading.Temperature);

        await aggregatesContainer.UpsertItemAsync(agg,
            new PartitionKey(agg.DeviceId));
    }
}

The change feed estimator

The estimator monitors processing lag — how far behind your processor is from the latest changes:

ChangeFeedProcessor estimator = monitoredContainer
    .GetChangeFeedEstimatorBuilder(
        processorName: "dashboardUpdater",
        estimationDelegate: HandleEstimation)
    .WithLeaseContainer(leaseContainer)
    .Build();

await estimator.StartAsync();

static async Task HandleEstimation(
    long remainingWork,
    CancellationToken ct)
{
    Console.WriteLine($"Estimated lag: {remainingWork} changes behind");

    if (remainingWork > 10000)
    {
        // Alert: processor is falling behind!
        // Consider scaling up instances or investigating bottlenecks
    }
}

Key details:

• remainingWork is the estimated number of unprocessed changes across all partitions
• The estimator reads from the same lease container as the processor
• It does NOT process changes — it only monitors
• Use it to trigger scaling decisions or alerting

💡 Exam tip: estimator vs processor

The change feed estimator and processor are separate components:

• The processor reads and processes changes
• The estimator monitors how far behind the processor is

They share the same lease container — the estimator reads lease checkpoints to calculate lag. You can run the estimator in a separate process (e.g., a monitoring Azure Function).

Pattern 4: Triggering downstream actions

The change feed can trigger actions in external systems:

Downstream Action	Example
Push notifications	Alert mobile app when sensor threshold exceeded
Search index update	Sync changes to Azure Cognitive Search
Analytics pipeline	Stream changes to Event Hubs or Kafka
Cache invalidation	Clear Redis or CDN cache when data changes
Audit log	Write change history to a separate container

🎬 Video walkthrough

🎬 Video coming soon

Change Feed Patterns — DP-420 Module 21

Change Feed Patterns — DP-420 Module 21

~16 min

Flashcards

Question

What is a materialized view in Cosmos DB?

Click or press Enter to reveal answer

Answer

A pre-computed, query-optimised container that's kept in sync with the source container via the change feed. It trades write cost (change feed processing + upserts) for read cost savings (cheap reads from the view).

Click to flip back

💡 Show hint

Question

What does the change feed estimator measure?

Click or press Enter to reveal answer

Answer

The estimated number of unprocessed changes (lag) across all partitions. It reads from the same lease container as the processor to calculate how far behind processing is. Use it for monitoring and scaling decisions.

Click to flip back

💡 Show hint

Question

How does denormalization propagation work with the change feed?

Click or press Enter to reveal answer

Answer

When the source of truth changes, the change feed processor finds all denormalized copies in other containers and updates them. This keeps embedded/copied data in sync across containers without manual coordination.

Click to flip back

💡 Show hint

Question

What resource does the change feed estimator share with the processor?

Click or press Enter to reveal answer

Answer

The lease container. The estimator reads lease checkpoint positions to calculate how many changes the processor hasn't yet processed. They use the same lease container but serve different purposes (processing vs monitoring).

Click to flip back

💡 Show hint

Knowledge Check

Knowledge Check

Amara's dashboard query scans 500M readings to find the latest value per device, costing 50,000 RU per execution. The dashboard refreshes every 30 seconds. What's the best optimisation?

AAdd a composite index on deviceId and timestampBUse the change feed to maintain a materialized view with one document per deviceCIncrease provisioned RU/s to handle the expensive queryDEnable the integrated cache to cache the query results

Check Answer

Knowledge Check

The change feed estimator reports remainingWork of 50,000 changes. What does this mean?

A50,000 changes have been processed successfullyBThere are approximately 50,000 unprocessed changes — the processor is behindCThe processor has failed on 50,000 changesDThe container has 50,000 total documents

Check Answer

Knowledge Check

GlobeCart embeds product names in order documents. When a product is renamed, which pattern keeps order documents in sync?

AStrong consistency ensures all containers see the latest dataBUse the change feed on the products container to update all order documents with the new nameCQuery all order documents periodically and compare product namesDStore a reference instead of embedding — always look up the latest product name

Check Answer

---

Next up: Monitoring — metrics, logs, alerts, and Azure Monitor insights for keeping your Cosmos DB solution healthy.