The problem
Are you surprised by the following error messages?
- Resource Microsoft.Databricks/workspaces [workspacename] failed with message : ResourceDeploymentFailure
- SubnetMissingRequiredDelegation
- Failed to prepare subnet. Please try again later?
Sure, I was too π
So, what the heck is going on?
May I ask if you have used the benefits of the VNet injection feature? Yes?! Well then… there is something you need to know.
- Azure Databricks developers have changed a way on how databricks vnet injection should be configured! (somewhere between September and October 2019)
- You NEED to modify your current scripts
- You NEED to modify your current workspaces if they were deployed using VNet injection during its “preview” period… Because, well, they will stop from working after January 31, 2020… Screen from docs:
That’s because VNet injection is now in GA (general availability) and of course preview features can change in one way or another without your consent π
And they have changed. That’s all π
What is VNet injection?
VNet injection in databricks is just a term for deploying Databricks in you desired VNet, created by you (or for you by your colleague).
Normally databricks are using its own VNet which is always automatically created behind the scenes. Let’s say that it is created just to make Databricks work out of the box π
But you may want to use VNet injection because of security reasons.
You have your own VNet with all settings and rules that guarantees secure access only for particular resources / subnets / connections etc.
So you deploy Databricks using VNet injection to just put it into particular VNet. That feature was publicly enabled for a while, but unfortunately was only in “preview”.
The solution
Well, fortunately it is well described in Databricks documentation π
I missed any updates on that, so I learned it hard way π
In a nutshell, you just need to edit your subnets used in VNet injection and apply subnet delegation. The documentation describes it in many ways (CLI, PowerShell, Azure Portal).
Then you need to edit your ARM scripts (if you have them) or any other deployment solution that was used to build your workspace (like Terraform) and apply delegation rule in your subnets to the service
Microsoft.Databricks-workspaces
Good luck! π
