TL;DR: Key Findings Every Databricks cluster node downloads over 12 GiB of data during startup. When routed via Azure Firewall (Data Exfiltration pattern), this can lead to monthly costs exceeding €3,000+ just for control plane traffic. For one bank’s project, this behavior caused over 100 TB of unexpected traffic throughCzytaj dalej / Read more
Category: Azure Databricks
Azure Databricks [EN]
Azure Databricks deployment and error: SubnetMissingRequiredDelegation – VNet injection problem
The problem Are you surprised by the following error messages? Resource Microsoft.Databricks/workspaces [workspacename] failed with message : ResourceDeploymentFailure SubnetMissingRequiredDelegation Failed to prepare subnet. Please try again later? Sure, I was too 🙂 So, what the heck is going on? May I ask if you have used the benefits of theCzytaj dalej / Read more
How to get Databricks WorkspaceID from REST API using PowerShell?
Getting information about ID of the Azure Databricks workspace is not so obvious. There is no information about it in the API specification: https://docs.databricks.com/api/latest/workspace.html That’s probably because this property is not related to the content of the workspace 🙂 That’s right! Raw JSON responses are intended to deliver different setCzytaj dalej / Read more