Tests are run twice which makes the build very slow

michael's Avatar

michael

22 Jan, 2020 09:19 PM

Hi there,

We realized that our build time had almost double over the past few months and did not have time to investigate until today.
This started after we upgraded our project structure to use the new .net project structure.

Please check our ci log out
https://ci.appveyor.com/project/open-systems-pharmacology-ci/pk-sim...

As you can see here, our tests are run once using the nunit runner

Discovering tests...OK
nunit3-console "C:\projects\pk-sim-g4na9\tests\PKSim.Matlab.Tests\bin\Debug\net472\PKSim.Matlab.Tests.dll" "C:\projects\pk-sim-g4na9\tests\PKSim.R.Tests\bin\Debug\net472\PKSim.R.Tests.dll" "C:\projects\pk-sim-g4na9\tests\PKSim.Tests\bin\Debug\net472\PKSim.Tests.dll" "C:\projects\pk-sim-g4na9\tests\PKSim.UI.Tests\bin\Debug\net472\PKSim.UI.Tests.dll" --where "cat != Reporting && cat != LocalOnly && cat != ProjectConverter && cat != NightlyOnly"
NUnit Console Runner 3.9.0
Copyright (c) 2018 Charlie Poole, Rob Prouse
Runtime Environment
   OS Version: Microsoft Windows NT 10.0.17763.0
  CLR Version: 4.0.30319.42000
Test Files
    C:\projects\pk-sim-g4na9\tests\PKSim.Matlab.Tests\bin\Debug\net472\PKSim.Matlab.Tests.dll
    C:\projects\pk-sim-g4na9\tests\PKSim.R.Tests\bin\Debug\net472\PKSim.R.Tests.dll
    C:\projects\pk-sim-g4na9\tests\PKSim.Tests\bin\Debug\net472\PKSim.Tests.dll
    C:\projects\pk-sim-g4na9\tests\PKSim.UI.Tests\bin\Debug\net472\PKSim.UI.Tests.dll
Test Filters
    Where: cat != Reporting && cat != LocalOnly && cat != ProjectConverter && cat != NightlyOnly
Tests Not Run
1) Ignored : PKSim.Core.When_resetting_the_simulation_settings_for_a_given_simulation.should_set_the_feced_concentration_as_non_persistable
47-8079 Observed data Amount in Urine / Feces cannot be mapped to observers in PK Sim
2) Ignored : PKSim.Core.When_resetting_the_simulation_settings_for_a_given_simulation.should_set_the_urine_concentration_as_non_persistable
47-8079 Observed data Amount in Urine / Feces cannot be mapped to observers in PK Sim
Run Settings
    DisposeRunners: True
    WorkDirectory: C:\projects\pk-sim-g4na9
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.7.2
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 2
Test Run Summary
  Overall result: Warning
  Test Count: 4729, Passed: 4727, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 2
    Skipped Tests - Ignored: 2, Explicit: 0, Other: 0
  Start time: 2020-01-22 20:58:37Z
    End time: 2020-01-22 21:06:41Z
    Duration: 484.531 seconds
Results (nunit3) saved as TestResult.xml

Then they are run again using donet core. Is there a way to prevent appveyor to run the tests using dotnet core?

Running .NET Core tests...dotnet test "C:\projects\pk-sim-g4na9\tests\PKSim.UI.Tests\PKSim.UI.Tests.csproj" --configuration Debug --no-build --filter "TestCategory!=Reporting&TestCategory!=LocalOnly&TestCategory!=ProjectConverter&TestCategory!=NightlyOnly"
Test run for C:\projects\pk-sim-g4na9\tests\PKSim.UI.Tests\bin\Debug\net472\PKSim.UI.Tests.dll(.NETFramework,Version=v4.7.2)
Microsoft (R) Test Execution Command Line Tool Version 16.3.0
...

Our appveyor config only has the following

test:
  categories:
    except:
      - Reporting
      - LocalOnly
      - ProjectConverter
      - NightlyOnly
How can I prevent the double execution of our tests?

Thanks

Michael

  1. Support Staff 1 Posted by Feodor Fitsner on 22 Jan, 2020 09:35 PM

    Feodor Fitsner's Avatar

    Hi Michael,

    I see from the provided log snippets that only PKSim.UI.Tests.dll is the one that double-tested, correct?

  2. 2 Posted by michael on 22 Jan, 2020 09:49 PM

    michael's Avatar

    Hi Feodor,

    No sorry i did not want to copy the whole output

    Here we go

    Running .NET Core tests...dotnet test "C:\projects\pk-sim-g4na9\tests\PKSim.UI.Tests\PKSim.UI.Tests.csproj" --configuration Debug --no-build --filter "TestCategory!=Reporting&TestCategory!=LocalOnly&TestCategory!=ProjectConverter&TestCategory!=NightlyOnly"
    Test run for C:\projects\pk-sim-g4na9\tests\PKSim.UI.Tests\bin\Debug\net472\PKSim.UI.Tests.dll(.NETFramework,Version=v4.7.2)
    Microsoft (R) Test Execution Command Line Tool Version 16.3.0
    Copyright (c) Microsoft Corporation.  All rights reserved.
    Starting test execution, please wait...
    A total of 1 test files matched the specified pattern.
    Test Run Successful.
    Total tests: 26
         Passed: 26
     Total time: 10.1022 Seconds
    dotnet test "C:\projects\pk-sim-g4na9\tests\PKSim.Matlab.Tests\PKSim.Matlab.Tests.csproj" --configuration Debug --no-build --filter "TestCategory!=Reporting&TestCategory!=LocalOnly&TestCategory!=ProjectConverter&TestCategory!=NightlyOnly"
    Test run for C:\projects\pk-sim-g4na9\tests\PKSim.Matlab.Tests\bin\Debug\net472\PKSim.Matlab.Tests.dll(.NETFramework,Version=v4.7.2)
    Microsoft (R) Test Execution Command Line Tool Version 16.3.0
    Copyright (c) Microsoft Corporation.  All rights reserved.
    Starting test execution, please wait...
    A total of 1 test files matched the specified pattern.
    Test Run Successful.
    Total tests: 15
         Passed: 15
     Total time: 18.3706 Seconds
    dotnet test "C:\projects\pk-sim-g4na9\tests\PKSim.Tests\PKSim.Tests.csproj" --configuration Debug --no-build --filter "TestCategory!=Reporting&TestCategory!=LocalOnly&TestCategory!=ProjectConverter&TestCategory!=NightlyOnly"
    Test run for C:\projects\pk-sim-g4na9\tests\PKSim.Tests\bin\Debug\net472\PKSim.Tests.dll(.NETFramework,Version=v4.7.2)
    Microsoft (R) Test Execution Command Line Tool Version 16.3.0
    Copyright (c) Microsoft Corporation.  All rights reserved.
    Starting test execution, please wait...
    A total of 1 test files matched the specified pattern.
      ! should_set_the_feced_concentration_as_non_persistable [2ms]
      ! should_set_the_urine_concentration_as_non_persistable
    Test Run Successful.
    Total tests: 4683
         Passed: 4681
        Skipped: 2
     Total time: 7.5591 Minutes
    dotnet test "C:\projects\pk-sim-g4na9\tests\PKSim.R.Tests\PKSim.R.Tests.csproj" --configuration Debug --no-build --filter "TestCategory!=Reporting&TestCategory!=LocalOnly&TestCategory!=ProjectConverter&TestCategory!=NightlyOnly"
    Test run for C:\projects\pk-sim-g4na9\tests\PKSim.R.Tests\bin\Debug\net472\PKSim.R.Tests.dll(.NETFramework,Version=v4.7.2)
    Microsoft (R) Test Execution Command Line Tool Version 16.3.0
    Copyright (c) Microsoft Corporation.  All rights reserved.
    Starting test execution, please wait...
    A total of 1 test files matched the specified pattern.
    Test Run Successful.
    Total tests: 5
         Passed: 5
     Total time: 14.8973 Seconds
    Build success
    
  3. Support Staff 3 Posted by Feodor Fitsner on 22 Jan, 2020 10:00 PM

    Feodor Fitsner's Avatar

    Define the following environment variable to disable .NET Core tests execution completely:

    APPVEYOR_BLOCK_DOTNETCORE_TESTS_AUTORUN: true
    
  4. 4 Posted by michael on 22 Jan, 2020 10:24 PM

    michael's Avatar

    Ah that's awesome. Sorry I did not find that as I was trying to disable the tests.
    Thanks for the prompt reply

  5. michael closed this discussion on 31 Jan, 2020 09:57 PM.

Comments are currently closed for this discussion. You can start a new one.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

Recent Discussions

07 Jul, 2020 06:21 PM
07 Jul, 2020 03:15 PM
05 Jul, 2020 02:33 AM
03 Jul, 2020 07:29 PM
03 Jul, 2020 03:53 AM

 

02 Jul, 2020 09:09 PM
02 Jul, 2020 03:24 PM
01 Jul, 2020 01:12 PM
30 Jun, 2020 04:26 PM
25 Jun, 2020 05:54 PM
24 Jun, 2020 08:11 AM