Migrate Batch to use local vnet, service principal, Entra ID auth etc.

[tamuri]

Azure Batch isn't able to size pools we need because errors when assigning IP addresses and limits of number of ports on the existing subnet. We're currently using Batch's classic networking and sharedkey auth but (I think) it's not suitable when we have many large pools. Mostly from reading the network troubleshooting.

My plan is:

• Switch pool communication mode to use simplified communication mode
• Create a new virtual network in tlo-b1-rg resource group, get the subnet id and put in pool specification (and disable public IP address configuration?? won't be able to connect to machines, but simpler config)
  • Check that it can still access container registry and storage account
• For Batch account tlob1ba to use virtual network subnet, need to switch from SharedKey auth to using Entra ID
  • Register new application (for unmanaged Batch control) with Entra
  • Setup a service principal
  • Set up the RBAC for the service principal for (at least) Batch account and VN (not sure about storage account?)
    • Didn't find a role to attach to network only - found contributor (too broad) and manage only (without connect).

Check that all works, then

• Put secrets in the key vault, check access again
• Merge into master
• Done!

cc @BinglingICL - sorry, a little bit of work to do for your runs

[tamuri]

Azure Batch isn't able to size pools we need because errors when assigning IP addresses

The error:

[BinglingICL]

Thanks Asif. It is great that issue and solution are found.