Transform Logs at Ingestion with DCRs Custom Tables and ASIM in Microsoft Sentinel

Every security telemetry source speaks its own dialect - a firewall, a SaaS audit log, and a Linux daemon all describe "a user did a thing from an address" with different field names, formats, and noise, and most of that raw volume is never queried yet is billed and retained all the same. Microsoft Sentinel solves this at two layers. At the bottom, Data Collection Rules (DCRs) apply an ingestion-time KQL transformation that parses raw records, drops fields you do not need, and routes the result into a custom table whose schema you control - shrinking cost and shaping data before a query runs. At the top, the Advanced Security Information Model (ASIM) is a set of normalized schemas, surfaced as query-time KQL parser functions, that let one analytics rule run cleanly across every source mapped to the schema.

In this lab, you act as a detection engineer onboarding a noisy custom log source. You will define a custom _CL table schema, build a Data Collection Endpoint and a Logs Ingestion DCR whose transformKql parses the raw payload and projects away the noise, and author and deploy an ASIM-aligned parser as a saved KQL function so the new source answers the same normalized queries as every built-in connector - treating the transform and parser as version-controlled engineering artifacts, not portal click-throughs.

Objectives

Upon completion of this advanced level lab, you will be able to:

• Create a custom _CL table with an explicit column schema using the Azure CLI
• Provision a Data Collection Endpoint (DCE) for ingestion-time processing
• Build a Logs Ingestion DCR with a stream declaration and a transformKql that parses, renames, and drops noisy columns before the custom table
• Author an ASIM-aligned normalization parser as a saved KQL function
• Verify the end-to-end pipeline - DCE, DCR, custom table, transform content, and ASIM parser all exist and are wired

Who is this lab for?

This lab is designed for:

• Security engineers mastering the data-engineering layer beneath detections
• Data engineers shaping and normalizing telemetry at ingestion
• Detection engineers managing transforms and ASIM parsers as code