Native configuration supports Azure low-priority or spot VMs

[adamrtalbot]

New feature

The Azure Batch autoscaling formula does not support low-priority or spot virtual machines. You can manually add them using a custom formula, but we could expose this as a simple option. See the Microsoft blog for a demonstration:

#4236

Usage scenario

Using low priority or spot machines.

Suggest implementation

Modify this block and replace $TargetDedicatedNodes with $TargetLowPriorityNodes when this option is enabled.

[pditommaso]

Horrible. Isn't there an option to control it via pool settings?

[pditommaso]

Look at PoolAddParameter

    /**
     * Set this property must not be specified if enableAutoScale is set to true. If enableAutoScale is set to false, then you must set either targetDedicatedNodes, targetLowPriorityNodes, or both.
     *
     * @param targetLowPriorityNodes the targetLowPriorityNodes value to set
     * @return the PoolAddParameter object itself.
     */
    public PoolAddParameter withTargetLowPriorityNodes(Integer targetLowPriorityNodes) {
        this.targetLowPriorityNodes = targetLowPriorityNodes;
        return this;
    }

nextflow/plugins/nf-azure/src/main/nextflow/cloud/azure/batch/AzBatchService.groovy

Lines 746 to 750 in 3c6409d

	final interval = spec.opts.scaleInterval.seconds as int
	poolParams
	.withEnableAutoScale(true)
	.withAutoScaleEvaluationInterval( new Period().withSeconds(interval) )
	.withAutoScaleFormula(scaleFormula(spec.opts))

[pditommaso]

Drafted an initial implementation. Likely the scaling formula needs to be updated when lowPriority is enabled.

[adamrtalbot]

Likely the scaling formula needs to be updated when lowPriority is enabled.

Correct. We 'just' need to use $TargetLowPriorityNodes instead of $TargetDedicatedNodes in the formula so shouldn't be too difficult: https://learn.microsoft.com/en-us/azure/batch/batch-automatic-scaling#read-write-service-defined-variables