Implement a Complexity Scoring mechanism for Bicep templates & deployments

[kilasuit]

Is your feature request related to a problem? Please describe.

Software deployed to Infrastructure with Bicep comes with a level of complexity in understanding all the moving parts that are required for a successful, maintainable & repeatable deployment process.

This becomes much more complex to understand with all the additional new language features as well as resource types that Bicep is capable of managing.

This level of knowledge is different for every person in every team across all differnt projects.

As such to aid in longer term maintainence it is required to have some kind of relatively simple, but configurable, scoring system to get a quick view at how complex a deployment could potentially be to understand in a manner that equates for the diverseness of the individuals in each of the teams that are working across these kinds of projects. It also helps aid in determining areas of knowledge strength as well as gaps.

Critically this should function without the need to initiate an actual deployment or any off device network calls, like to items including templates that are remotely hosted like via Bicep Modules, Template Specs, ARM Templates that are stored in a Storage Account, or are web accessible via either anon or authenticated access & should preference analyzing local file system templates.

Describe the solution you'd like

I created some years ago this PowerShell Module - ARMTemplateComplexity that started this work and provides some flexibility in attempting to determine a Complexity Score for a ARM Template or a repository of ARM Templates. This module is Available on the PowerShell Gallery and has not been updated since the initial release.

Prior to developing this I had previously used a similar mental model that provided a weighted points scoring system depending on factors like resource & schema version familiarity for any resources in a template, along with any possible cross dependencies when looking to deploy resources across the many different logical boundaries, Region, Resource Group, Subscription, Management Group or even across Tenants

This enabled me to calculate quickly a number of project specific data points that helped with managing different kinds of project workloads, including across teams & orgs.

I also used this as a way of determining whether there were potential optimisations in an overall deployment sequence across the codebase for complex Azure deployments, typically deployed via a CI/CD Pipeline from 1 or more repositories.

This also could including those as part of existing open source repositories like the Azure-QuickStart-Templates and CAF & WAF, whilst also enabling me ability to demonstrate the flexibility & benefits of authoring templates using ARM templates over the other similar automation tooling that was available at the time, like az cli Azure PowerShell, SDK, or 3rd party offerings.

Ideally, I would like to see my initial project expanded from what I have published and for this to be natively available as part of the bicep cli tool, perhaps as a seperate bicep command like bicep analyze which will allow passing a configuration of what a base score may look like whether that be mapped at an indivual level, team level, project level, etc to a provide point in time reference that can be rescored anytime during the development lifecyle of any templates, the resources & schema definitions as well as any new language features whilst bicep evolves.

Notes

This probably can be expanded on with projects like Radius, Dapr or with languages & libaries like those in the dotnet ecosystem etc, in such a way that provides a way to get a rough estimate of the costs (particularly mental strain and the time required to analyse) that may be incurred whilst performing deep analytical understanding of 1 or more isolated or interlinked application / deployment sequences that may or may not use bicep templates, especially if you want to estimate if there are any longer term maintainence improvements or optimisations that may possible when new features are released or deprecated.

[shenglol]

Thanks for the detailed write-up. Totally agree that large Bicep deployments can be hard to understand, especially with many nested modules, custom types, and advanced language features involved.

A built-in scoring system, however, might be tricky because the weights and definition of “complexity” may vary between teams, projects, and even individuals. I'm not sure if it's proper for Bicep CLI to provide a single, opinionated score that fits everyone.

Personally, I think what we can do is provide the building blocks. We can consider adding a bicep summarize command that would output objective statistics for a Bicep file and all its dependencies, such as:

• Number of user-defined types, resources, modules, parameters, variables, outputs
• Number of resource properties
• Number of expressions
• Module nesting depth
• Totals across the full dependency graph

External tools could then apply whatever scoring model makes sense without requiring Bicep itself to pick one.