Experimental: Update to Microsoft Testing Platform v2

[bart-vmware]

Description

This PR migrates existing xUnit v3 tests to run on Microsoft Testing Platform v2, instead of VSTest. This enables running tests both via dotnet test and as direct executables (the latter hasn't been tried).

This PR currently references preview packages from the https://github.com/microsoft/testfx dev-feed. At its core, everything can be made to work almost as well as VSTest. See the list of limitations below. Given that running tests is the primary quality gate for our work and that there's no advantage in switching to the not-so-polished-yet MTPv2, we're holding off on the update for now. The biggest limitation is degraded logging in CI-build: it's not easy to spot which test(s) failed and why.

What's changed

• Replaced coverlet with Microsoft.Testing.Extensions.CodeCoverage, because it is incompatible
  • Removed Coverlet workaround
• Removed xunit.runner.visualstudio and Microsoft.NET.Test.Sdk, which are no longer needed
• Alternate libraries used for hang/crashdump and TRX reports
• Added global.json, which is required for MTP v2 when using the .NET 10 SDK
• Different file format for coverage settings (.runsettings replaced by coverage.config)
• Added workaround to get line/branch coverage in Sonar
  • The Microsoft.Testing.Extensions.CodeCoverage coverage collector only supports the formats Cobertura and Visual Studio XML (the latter doesn't contain branch coverage information)
  • Sonar needs OpenCover, but that format is no longer available
  • We use ReportGenerator in GHA workflow to convert from Cobertura to Sonar private file format
  • Because Sonar is incompatible with deterministic paths, and MTP barely provides command-line parameters, we need a second configuration file (coverage.sonar.config)
• Different command-line arguments to take crash/hangdumps and to filter tests (caution: multiple filter parameters are OR-ed together!)
• Make xUnit use MTP2 via UseMicrosoftTestingPlatformRunner
• Keep using xunit.runner.json instead of testconfig.json, because it works with multiple runners
• Not using the superior GitHubActionsTestLogger, due to license terms about Russia/Ukraine, which we can't enforce on our users
• When no tests are found, dotnet test exists with an error (can be overruled with --minimum-expected-tests 0 or catching the related exit code)
• To collect coverage locally:

  dotnet tool install dotnet-reportgenerator-globaltool
  dotnet test --project .\src\Common\test\Common.Test\Steeltoe.Common.Test.csproj --configuration Release --coverage --report-trx --coverage-settings coverage.config --results-directory TestOutput /p:ContinuousIntegrationBuild=true
  dotnet reportgenerator -reports:TestOutput/*.xml -targetdir:TestOutput/CoverageOutput
  

• The CI build log does not tell which test(s) failed (only the assembly, without target framework), and there is no stacktrace. You need to search for it on the GitHub Actions summary tab

Known issues

• MTP: Add command-line parameters for settings in coverage configuration file microsoft/codecoverage#199
• Add option to collect gcdump (MTPv2) microsoft/testfx#7126
• MTPv2: clean output in GitHub Actions? microsoft/testfx#7056 / Add test counts to per-assembly summaries dotnet/sdk#52128 / [MTP] CI logs unnecessarily long and hard to read microsoft/testfx#5712
• Excessive output when crashdump kicks in microsoft/testfx#7045
• VS Test Explorer never completes when debugging test in WSL2 microsoft/testfx#7043
• Add test for hangdump for dotnet test microsoft/testfx#6859 (merged but not yet released, requires dev-feed)
• When HangDump extension triggers, test run does not produce a .trx file microsoft/testfx#6778
• [MTP]: Timeout doesn't abort hanging tests microsoft/testfx#6776
• [Rant] All the "wheel reinvention" going on in Testing.Platform is extremely disappointing microsoft/testfx#5497 / Testing.Platform should not be a blackbox: leveraging standard Hosting model and Program.cs for test application setup microsoft/testfx#4198
• CTRL+C does not complete the test run when there is long running test microsoft/testfx#5345 / Handle SIGTERM shutdown signal to fire the cancellation microsoft/testfx#2035
• No failing test details with MTP and dotnet test support with xunit v3 microsoft/testfx#5188
• [MTP] Better message for never finished tests? microsoft/testfx#5170
• Testing Platform overwriting/removing user console output microsoft/testfx#4425

Verification

• Tests are discovered (5520 in total, 1840 per target framework)
  • In Visual Studio 2026 Test Explorer
  • In Visual Studio 2026 Test Explorer on WSL2 with Ubuntu-22.04
  • In Visual Studio 2026 R# Unit Tests
  • In dotnet test on the command line
• Tests can be run
  • In Visual Studio 2026 Test Explorer
  • In Visual Studio 2026 Test Explorer on WSL2 with Ubuntu-22.04
  • In Visual Studio 2026 R# Unit Tests
  • In dotnet test on the command line
  • Test outcomes are listed on the GitHub Actions summary tab
• Tests can be debugged
  • In Visual Studio 2026 Test Explorer
  • In Visual Studio 2026 Test Explorer on WSL2 with Ubuntu-22.04
    ✋ It's only possible to debug once; a VS restart is required to debug again (tracked at VS Test Explorer never completes when debugging test in WSL2 microsoft/testfx#7043)
  • In Visual Studio 2026 R# Unit Tests
• All tests succeed
  • In Visual Studio 2026 Test Explorer
  • In Visual Studio 2026 Test Explorer on WSL2 with Ubuntu-22.04
  • In Visual Studio 2026 R# Unit Tests
  • In dotnet test on the command line
  • In CI build on Ubuntu/Windows/macOS, split into regular/memorydump testruns
• Coverage is reported
  • Coverage files contain source-link paths (except for Sonar)
  • Coverage on PR diff in SonarCloud and Sonar PR comment
    
  • Coverage on full branch displayed on GitHub Actions summary tab and PR comment (only Steeltoe libraries)
    
  • Coverage on full branch roughly matches outcome from main branch (only Steeltoe libraries)
    
• Test failures
  • Crashdump is captured and uploaded during CI build
    ✋ Works, but outputs a 160 MB text log, which makes the GitHub Actions UI completely unresponsive; tracked at Excessive output when crashdump kicks in microsoft/testfx#7045
  • Hangdump is captured and uploaded during CI build
  • Failed test is reported
    • In Visual Studio 2026 Test Explorer
    • In Visual Studio 2026 Test Explorer on WSL2 with Ubuntu-22.04
    • In Visual Studio 2026 R# Unit Tests
    • In dotnet test on the command line
    • On GitHub Actions summary tab
    • In CI build log
  • Skipped test is reported
    • In Visual Studio 2026 Test Explorer
    • In Visual Studio 2026 Test Explorer on WSL2 with Ubuntu-22.04
    • In Visual Studio 2026 R# Unit Tests
    • In dotnet test on the command line
    • On GitHub Actions summary tab
    • In CI build log

Quality checklist

• Your code complies with our Coding Style.
• You've updated unit and/or integration tests for your change, where applicable.
• You've updated documentation for your change, where applicable.
  If your change affects other repositories, such as Documentation, Samples and/or MainSite, add linked PRs here.
• There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.
• You've added required license files and/or file headers (explaining where the code came from with proper attribution), where code is copied from StackOverflow, a blog, or OSS.

[github-actions]

Summary - All Code Coverage (ubuntu-latest)

Assembly	Line coverage	Branch coverage
Steeltoe.Bootstrap.AutoConfiguration	97.4%	100%
Steeltoe.Common	84.5%	77.7%
Steeltoe.Common.Certificates	96.2%	85.7%
Steeltoe.Common.Hosting	84%	65%
Steeltoe.Common.Http	100%	82.3%
Steeltoe.Common.Logging	81.3%	41.6%
Steeltoe.Common.Net	64.5%	66.6%
Steeltoe.Configuration.Abstractions	98.5%	91.6%
Steeltoe.Configuration.CloudFoundry	99.1%	92.3%
Steeltoe.Configuration.ConfigServer	97.4%	92.2%
Steeltoe.Configuration.Encryption	97.7%	92.4%
Steeltoe.Configuration.Kubernetes.ServiceBindings	93.3%	87.5%
Steeltoe.Configuration.Placeholder	91.9%	78.8%
Steeltoe.Configuration.RandomValue	93.2%	87.5%
Steeltoe.Configuration.SpringBoot	98.3%	95%
Steeltoe.Connectors	94.3%	88.7%
Steeltoe.Connectors.EntityFrameworkCore	81.5%	75%
Steeltoe.Discovery.Configuration	92.8%	100%
Steeltoe.Discovery.Consul	97.8%	96.2%
Steeltoe.Discovery.Eureka	91.5%	84.5%
Steeltoe.Discovery.HttpClients	95%	95.8%
Steeltoe.Logging.Abstractions	99.4%	97%
Steeltoe.Logging.DynamicConsole	100%	96.8%
Steeltoe.Logging.DynamicSerilog	99.1%	95.8%
Steeltoe.Management.Abstractions	100%	100%
Steeltoe.Management.Endpoint	95.5%	89.3%
Steeltoe.Management.Prometheus	95.9%	92.3%
Steeltoe.Management.Tasks	100%	****
Steeltoe.Management.Tracing	100%	83.3%
Steeltoe.Security.Authentication.JwtBearer	100%	100%
Steeltoe.Security.Authentication.OpenIdConnect	74.4%	57.6%
Steeltoe.Security.Authorization.Certificate	96.3%	80%
Steeltoe.Security.DataProtection.Redis	100%	****

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
1 Security Hotspot
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud