Test instabilities

Christian Soltenborn's Avatar

Christian Soltenborn

20 Mar, 2017 10:23 AM

Hi,

a couple of weeks ago I reported that our end-to-end tests would be rather instable with image VS 2017 RC (at that time). Feodor seemed to have fixed that problem, but now I'm seeing instable tests again.

The following two builds do build the same commit, but with different outcomes:
* 709 * 710

Interestingly, the 710 build job on the VS 2017 image failed due to some weird VS exceptions:

System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at Microsoft.VisualStudio.Telemetry.TelemetryContext.FlushPostedProperties()
   at Microsoft.VisualStudio.Telemetry.TelemetryTaskTimer.<>c__DisplayClass11_0.<Start>b__0(Task previousTask)
   at System.Threading.Tasks.ContinuationTaskFromTask.InnerInvoke()

Would you be so kind to check out once more what's going on there?

Thanks in advance
Christian

  1. 1 Posted by Christian Solte... on 20 Mar, 2017 01:28 PM

    Christian Soltenborn's Avatar

    In case you haven't seen build 711: Same commit, build passed this time, leaving us with three builds and three different build results...

  2. Support Staff 2 Posted by Feodor Fitsner on 20 Mar, 2017 02:21 PM

    Feodor Fitsner's Avatar

    I see "Visual Studio 2015" job is failing. So you see instability on VS 2015 image or VS 2017 - could you please clarify? Also, when was the last good build?

  3. 3 Posted by Christian Solte... on 21 Mar, 2017 10:38 AM

    Christian Soltenborn's Avatar

    Sorry, that was indeed a bit confusing... I've looked closely at our build results once more:

    • Main cause for failing builds is the test that checks whether our adapter's SettingsProvider has been registered successfully. This is very comparable to the test which used to fail last time and got fixed by your patch. However, this happens also on VS2015 (although mostly on VS2017) Builds: 710, 709, 706, 705, 697, 683
    • Some builds are failing due to a failed timing test (I suspect that this happens if the build machine we are running on is rather busy - probably not much we can do about that despite increasing the test's timing tolerance)
    • weird AccessViolationException exception on VS 2017 as described above (build 710)
    • last good build was (after a couple of failed builds) build 711 (same commit as 706, 708-710)

    Thus, it looks like nothing has changed from your side - sorry for wasting your time. We still have no idea why our SettingsProvider test currently fails regularly (and this never happens on our local machines), but it's not a very new problem.

    If we get more AccessViolationExceptions, I'll let you know.

  4. Support Staff 4 Posted by Feodor Fitsner on 21 Mar, 2017 10:20 PM

    Feodor Fitsner's Avatar

    Sure, is there anything we could (should) do on VS 2017 to make your tests work please let us know. Also, looking via RDP into build worker could help you troubleshoot many issues - RDP works for VS 2017 image now.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac