Home → Problems →

Tests of VS test adapter fail if VsTestConsole.exe's `/Enablecodecoverage` is set

• Edit

Christian Soltenborn

Apr 15, 2016 @ 05:26 AM

Hi,

we are developing a VS test adapter, here is our build.

As part of our build, we have end-to-end-tests which invoke VsTest.Console.exe with our adapter (by using the /UseVsixExtensions:true as well as the /TestAdapterPath:<dir> command line options). We do that with several combinations of other command line parameters, including the /Enablecodecoverage parameter.

Now, since a couple of days, all tests making use of that parameter (and only those tests) fail - VsTestConsole.exe is apparantly not able to find tests any more if the parameter is set. The last build where this worked is 355 13 days ago, followed by build 356 triggered by a pull request 9 days ago. However, yesterday I pushed something to our repository, and to my surprise, the build failed. I triggered rebuilding the commit of build 355, and it now also fails (see build 358).

Thus, my assumption is that something has changed with the OS we're running our build on, which is Visual Studio 2015 in our case. Any idea what's going on?

Here are some links for your convience: our history of builds, appveyor.yml.

Thanks in advance (as always)
Christian

1. Support Staff 1 Posted by Feodor Fitsner on Apr 15, 2016 @ 05:41 AM

   

   Hi Christian,

   Yes, Visual Studio 2015 Update 2 was deployed 3 days ago: http://www.appveyor.com/updates/2016/04/11

   VS 2015 Update 2 was RTMed and it's supposed to be a "painless" in-place update, but looks like some things changed. I don't know how your tests affected though. Maybe logging via RDP will help you to troubleshoot the issue: http://www.appveyor.com/docs/how-to/rdp-to-build-worker

   • Edit

2. 2 Posted by Christian Solte... on Apr 15, 2016 @ 05:47 AM

   

   Hi Feodor,

   thanks for the fast answer! What's weird is that I did the upgrade on my local machine, too, and the tests are running fine locally.

   I will have a look at RDP execution (I forgot to mention that build time went up from roughly 15mins to 55mins, so maybe there's indeed something to observe) - I'll let you know the results.

   All the best
   Christian

   • Edit

3. 3 Posted by Christian Solte... on Apr 15, 2016 @ 06:06 AM

   

   Hi again,

   I just had a quick look, but there's nothing to see (to be honest, I do not really know what to look for, since no UI is involved when our build is running, but at least there's nothing obvious like a dialog popping up in context of a failing test)

   Any other idea on what I could try, or any hints on what I could inspect on the build machine?

   All the best
   Christian

   • Edit

4. 4 Posted by Christian Solte... on Apr 15, 2016 @ 01:38 PM

   

   Hi Feodor,

   just a quick update: I fixed our build by changing the OS to Previous Visual Studio 2015. However, my fear is that that OS will be updated as soon as Update 3 of VS is released (which will probably then be the base for OS Visual Studio 2015, since I assume that the Previous... OS will always be one step behind the actual OS!?

   So let's keep fingers crossed that if that happens, our build will run on Update 3...

   Thanks anyways for your assistance (and of cource, feel free to add any thoughts on that issue you might have).

   All the best
   Christian

   • Edit

5. Support Staff 5 Posted by Feodor Fitsner on Apr 16, 2016 @ 11:23 PM

   

   Hi Christian,

   Build worker image has been fixed and your tests are running fine now. Visual Studio Agents 2015 with Update 2 was the missing point. Thank you!

   • Edit

6. 6 Posted by Christian Solte... on Apr 18, 2016 @ 08:39 AM

   

   Hi Feodor,

   awesome - thanks once more for your efforts!

   All the best
   Christian

   • Edit

7. Ilya Finkelshteyn closed this discussion on Aug 25, 2018 @ 02:05 AM.