nunit3-console runner has changed

rob's Avatar

rob

29 Feb, 2024 04:50 PM

We suddenly had an issue where our coverage results were no longer being sent to codecov.io and our GitHub badges are reading 0% coverage.

After reading the logs and reproducing the issue I was able to narrow this down to the change of install_nunit3.ps1 when the version updated but also the source changed from a zip file to nupkg file. As I browse through the respective files it seems the nupkg versions do not include the addins for nunit3, where the zip files did, so our chosen method of running the nunit3-console against the csproj files is not supported anymore.

https://github.com/appveyor/build-images/pull/145/files#diff-5c38574ba98260bb7db2f0a98419116cbb2ca2fa79e8f2c9e2447a22c1bb1eefR14

Should I manually install the zip version somehow to restore full functionality?

  1. Support Staff 1 Posted by Owen McDonnell on 29 Feb, 2024 10:25 PM

    Owen McDonnell's Avatar

    Thanks for bringing this to our attention though I think the lack of files has less to do with the package format and more to do with the fact that the PR you linked to is installing the ConsoleRunner which lacks the extensions included with the Console.
    In any case, I'm still not sure if this is related to your problem since that change was made for a custom image and public images have not been updated since. What build image are you using?

  2. 2 Posted by rob on 29 Feb, 2024 10:27 PM

    rob's Avatar

    Using from yaml:

    image: Visual Studio 2019

    running nunit3-console.exe and we see around this time a change from 3.15.2 to 3.16.3.

  3. 3 Posted by rob on 29 Feb, 2024 10:38 PM

    rob's Avatar

    On Dec. 2 we saw this version:

    Launching OpenCover 4.7.1221.0
    Executing: C:\Tools\NUnit3\bin\nunit3-console.exe
    NUnit Console 3.15.2 (Release)
    Copyright (c) 2022 Charlie Poole, Rob Prouse
    Sunday, December 3, 2023 1:01:43 AM

    On December 9th during the next run we see

    Launching OpenCover 4.7.1221.0
    Executing: C:\Tools\NUnit3\bin\nunit3-console.exe
    NUnit Console 3.16.3 (Release)
    Copyright (c) 2022 Charlie Poole, Rob Prouse
    Sunday, December 10, 2023 1:01:44 A

  4. Support Staff 4 Posted by Owen McDonnell on 29 Feb, 2024 11:00 PM

    Owen McDonnell's Avatar

    Ah, you're right, my mistake. We did deploy a specific update for that.
    I've added the console with extensions to next image update to do list, which I will begin on after Ubuntu image updates. In the meantime, you could just manually install 3.17.0 for example, from the same script, i.e.

    [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12


    $nunitPath = "$env:SYSTEMDRIVE\Tools\NUnit3"


    if (Test-Path $nunitPath) { Remove-Item $nunitPath -Recurse -Force }


    $zipPath = "$env:TEMP\NUnit.Console-3.15.2.zip" $tempPath = "$env:TEMP\NUnit.Console" (New-Object Net.WebClient).DownloadFile('https://github.com/nunit/nunit-console/releases/download/3.17.0/NUnit.Console-3.17.0.zip', $zipPath) 7z x $zipPath -y -o"$tempPath" | Out-Null New-Item -Path "$nunitPath" -ItemType Directory -Force | Out-Null [IO.Directory]::Move("$tempPath\bin\net35", "$nunitPath\bin") Copy-Item -Path "$tempPath\bin\agents" -Destination $nunitPath -Recurse Remove-Item $zipPath
  5. 5 Posted by rob on 01 Mar, 2024 12:27 PM

    rob's Avatar

    How long until the next update? Our tests are running and builds are passing so if it's soon, we could wait for the coverage reports to be restored.

  6. Support Staff 6 Posted by Owen McDonnell on 01 Mar, 2024 05:54 PM

    Owen McDonnell's Avatar

    We're hoping to deploy new Ubuntu images early next week, after which we'll start on Visual studio images. So, if all goes well, the following week.

  7. Support Staff 7 Posted by Owen McDonnell on 14 Mar, 2024 04:31 AM

    Owen McDonnell's Avatar

    Just an update. New VS images are being worked on now but were started a little later due to some complications in new Ubuntu images release.
    They will hopefully be released sometime next week.

  8. 8 Posted by rob on 22 Mar, 2024 12:07 PM

    rob's Avatar

    Can you let me know when this is complete? Our code coverage has not come back yet, but I will wait to do any more investigation until I know the images are updated with different nunit3-console from zip file.

  9. Support Staff 9 Posted by Owen McDonnell on 22 Mar, 2024 09:02 PM

    Owen McDonnell's Avatar

    Sorry, we had to to allocate some time to some other issues that cropped up. I will update here when we have new images.

  10. Support Staff 10 Posted by Owen McDonnell on 31 Mar, 2024 06:04 AM

    Owen McDonnell's Avatar

    Sorry again for the delay. We have released a new Visual studio 2019 image that hopefully resolves your nunit issue. Visual studio 2022 image is now being updated, after which, documentation for new images will follow.
    Let us know if it works for you.

  11. Support Staff 11 Posted by Owen McDonnell on 03 Apr, 2024 06:35 AM

    Owen McDonnell's Avatar

    Both Visual studio 2019 and 2022 images with updated NUnit3 are now live.
    Documentation update PR here

  12. 12 Posted by rob on 05 Apr, 2024 10:39 AM

    rob's Avatar

    Thanks for the update.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac