I have a below VsTest task in my azure pipeline. It runs more than 5 thousand tests. Does someone knows what exactly the ‘customBatchSizeValue’ means ? I know the tests are grouped as batches, each containing 800 test cases and for every batch, it generates a separate trx file and a test run. My question is: do the 800 test cases in every batch run parallelly as well ? How does it changes when tests are UI & non-UI ?
I already referred this MS doc on VsTest, but doesn’t seem to be detailed. https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vstest-v2?view=azure-pipelines
- task: VSTest@2
displayName: 'Run UI tests'
inputs:
testAssemblyVer2: '**UITest.dll'
searchFolder: '$(TargetDir)UITest'
vsTestVersion: toolsInstaller
runSettingsFile: '$(TargetDir)UITest.pipeline.runsettings'
pathtoCustomTestAdapters: 'C:TempTestAdapter'
runTestsInIsolation: false
batchingBasedOnAgentsOption: customBatchSize
customBatchSizeValue: 800
testRunTitle: 'UITest from Directory (1)'
platform: x64
failOnMinTestsNotRun: true
rerunFailedTests: true
continueOnError: true
timeoutInMinutes: 180