Home → Questions →

nunit3-console runner has changed

• Edit

rob

29 Feb, 2024 04:50 PM

We suddenly had an issue where our coverage results were no longer being sent to codecov.io and our GitHub badges are reading 0% coverage.

After reading the logs and reproducing the issue I was able to narrow this down to the change of install_nunit3.ps1 when the version updated but also the source changed from a zip file to nupkg file. As I browse through the respective files it seems the nupkg versions do not include the addins for nunit3, where the zip files did, so our chosen method of running the nunit3-console against the csproj files is not supported anymore.

https://github.com/appveyor/build-images/pull/145/files#diff-5c38574ba98260bb7db2f0a98419116cbb2ca2fa79e8f2c9e2447a22c1bb1eefR14

Should I manually install the zip version somehow to restore full functionality?

1. Support Staff 1 Posted by Owen McDonnell on 29 Feb, 2024 10:25 PM

   

   Thanks for bringing this to our attention though I think the lack of files has less to do with the package format and more to do with the fact that the PR you linked to is installing the ConsoleRunner which lacks the extensions included with the Console.
   In any case, I'm still not sure if this is related to your problem since that change was made for a custom image and public images have not been updated since. What build image are you using?

   • Edit

2. 2 Posted by rob on 29 Feb, 2024 10:27 PM

   

   Using from yaml:

   image: Visual Studio 2019

   running nunit3-console.exe and we see around this time a change from 3.15.2 to 3.16.3.

   • Edit

3. 3 Posted by rob on 29 Feb, 2024 10:38 PM

   

   On Dec. 2 we saw this version:

   Launching OpenCover 4.7.1221.0
   Executing: C:\Tools\NUnit3\bin\nunit3-console.exe
   NUnit Console 3.15.2 (Release)
   Copyright (c) 2022 Charlie Poole, Rob Prouse
   Sunday, December 3, 2023 1:01:43 AM

   On December 9th during the next run we see

   Launching OpenCover 4.7.1221.0
   Executing: C:\Tools\NUnit3\bin\nunit3-console.exe
   NUnit Console 3.16.3 (Release)
   Copyright (c) 2022 Charlie Poole, Rob Prouse
   Sunday, December 10, 2023 1:01:44 A

   • Edit

4. Support Staff 4 Posted by Owen McDonnell on 29 Feb, 2024 11:00 PM

   

   Ah, you're right, my mistake. We did deploy a specific update for that.
   I've added the console with extensions to next image update to do list, which I will begin on after Ubuntu image updates. In the meantime, you could just manually install 3.17.0 for example, from the same script, i.e.
   

   [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
   
   
   
   
   $nunitPath = "$env:SYSTEMDRIVE\Tools\NUnit3"
   
   
   
   
   if (Test-Path $nunitPath) {
       Remove-Item $nunitPath -Recurse -Force
   }
   
   
   
   
   $zipPath = "$env:TEMP\NUnit.Console-3.15.2.zip"
   $tempPath = "$env:TEMP\NUnit.Console"
   (New-Object Net.WebClient).DownloadFile('https://github.com/nunit/nunit-console/releases/download/3.17.0/NUnit.Console-3.17.0.zip', $zipPath)
   7z x $zipPath -y -o"$tempPath" | Out-Null
   New-Item -Path "$nunitPath" -ItemType Directory -Force | Out-Null
   [IO.Directory]::Move("$tempPath\bin\net35", "$nunitPath\bin")
   Copy-Item -Path "$tempPath\bin\agents" -Destination $nunitPath -Recurse
   Remove-Item $zipPath
   

   • Edit

5. 5 Posted by rob on 01 Mar, 2024 12:27 PM

   

   How long until the next update? Our tests are running and builds are passing so if it's soon, we could wait for the coverage reports to be restored.

   • Edit

6. Support Staff 6 Posted by Owen McDonnell on 01 Mar, 2024 05:54 PM

   

   We're hoping to deploy new Ubuntu images early next week, after which we'll start on Visual studio images. So, if all goes well, the following week.

   • Edit

7. Support Staff 7 Posted by Owen McDonnell on 14 Mar, 2024 04:31 AM

   

   Just an update. New VS images are being worked on now but were started a little later due to some complications in new Ubuntu images release.
   They will hopefully be released sometime next week.

   • Edit

8. 8 Posted by rob on 22 Mar, 2024 12:07 PM

   

   Can you let me know when this is complete? Our code coverage has not come back yet, but I will wait to do any more investigation until I know the images are updated with different nunit3-console from zip file.

   • Edit

9. Support Staff 9 Posted by Owen McDonnell on 22 Mar, 2024 09:02 PM

   

   Sorry, we had to to allocate some time to some other issues that cropped up. I will update here when we have new images.

   • Edit

10. Support Staff 10 Posted by Owen McDonnell on 31 Mar, 2024 06:04 AM

    

    Sorry again for the delay. We have released a new Visual studio 2019 image that hopefully resolves your nunit issue. Visual studio 2022 image is now being updated, after which, documentation for new images will follow.
    Let us know if it works for you.

    • Edit

11. Support Staff 11 Posted by Owen McDonnell on 03 Apr, 2024 06:35 AM

    

    Both Visual studio 2019 and 2022 images with updated NUnit3 are now live.
    Documentation update PR here

    • Edit

12. 12 Posted by rob on 05 Apr, 2024 10:39 AM

    

    Thanks for the update.

    • Edit