Event Triggers – Does it match or not?

Posted in : Azure, Azure, Microsoft, Office 365, Other, Powershell, Windows, Windows Server By Johan Nilsson Translate with Google ⟶

3 months ago

Our way of keeping track of Azure Costs are heavily based on the cost export from Azure Cost Management Exports. These cost exports are based on Daily- and Monthly Actual Cost & Daily- and Monthly Amortized Cost. The exports are more or less the same.  An ordinary Azure Cost export, generating and exporting a CSV file to a Azure Storage Account (ADLS2) and both of them are connected to two completely identical ”BlobEventsTriggers”. But for some magical reason, the behaviour of the Actual Cost exports and the Amortized Cost exports differs. In this particular case we will be looking at ”Daily Actual Cost” and ”Daily Amortized Cost” and their triggers. The only difference between the two triggers above is that the first one (Actual Daily Cost) exports from the Root Management Group scope, while the second one (Daily Amortized Cost) exports from a specific Billing Account. Both cost exports are generating a file every morning, but only one of them is actually triggering the pipelines they are connected to.

The worst part of this whole shenanigan is that if you download the generated Cost Export CSV file and manually upload it the dysfunctional trigger actually works. 

Trigger 1: DailyActualCost

IMG1: Event Trigger

IMG2: Event Trigger files

Trigger 2 : DailyAmortizedCost

IMG1: Event Trigger

IMG2: Event Trigger files

Both of the triggers are created in the exact same way through Azure Synapse and have received the same Event Grid Filters. As shown in the images below where image 1 (Daily Actual Cost) generates a matched event, while image 2 (Daily Amortized Cost) does not.

IMG1: Actual Cost Event

IMG2: Amortized Cost Event

By looking at  these two events, you can see that 2/2 ”Advanced Filter Evaluations (Sum)” are resulting in 1 ”Matched Event (Sum)”, while the second image indicated that only 1/2 ”Advanced Filter Evaluations (Sum)” are met, thus not resulting in a ”Matched Event (Sum)”.  Since the ”Advanced Filter Evaluations (Sum)” for some reason differs between the two triggers this indicates that one of the two filters are not met.  As shown in the image below, both Event Grids have the same two filters. The first one ”data.api” and the second one ”data.contentLength”.

For more information about these Event properties, you can check out this this Microsoft Learn page to deep dive and fully understand all the different advanced filters and how they work in Azure Blob Storage as Event Grid sources work.

I first tried to remove the ”data.api” key from the Amortized Cost Daily trigger without any success, and then I removed the ”data.contentLength” key from the same event and it magically started working. Thus indicating that the non matching filter is the ”data.contentLength” key. I also reinstated the ”data.api” keys and verified that it still works

I cannot explanation the ”data.contentLength” key behaviour, and why it does not register any bytes in the Daily Amortized Cost, it just doesn’t. I have created a case with Microsoft to get an explanation, but yet to receive an explanation. Until then, if anyone reading this can clarify, or just want to say hello, feel free to email me at johan.nilsson@xenit.se 

Tags : Actual cost, ADF, ADLS2, Advanced filter, Amortized cost, Azure cost, Azure cost export, Azure Pipelines, Azure Storage, Azure Storage Account, Cost export, Data factory, Data lake, Data lake gen 2, data.api, data.contentlength, Event grid, Event triggers, Matched events, Storage, Synapse, Synapse Pipeline, Synapse Workspace