tag:help.appveyor.com,2012-11-13:/discussions/problems/570-selenium-testsAppVeyor: Discussion 2018-08-25T01:50:18Ztag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T15:49:13Z2014-07-21T15:56:48ZSelenium Tests<div><p>I'm pushing on to run my Selenium tests.</p>
<p>I have a web site to test and a selenium test project with the
tests.</p>
<p>I build and deploy the test site locally. I then run the
selenium tests under mstest.</p>
<p>The good part is that the tests run and report correctly.</p>
<p>The problem is that the build fails. As the tests run I get a
lot of 'noise' in the console and this includes some errors from
Selenium (I googled and they appear to be known bugs). These don't
cause the tests to fail however they're being flagged in red in the
console and I assume they are what is causing the build to fail (it
passes without the tests being run). Is there some way to prevent
these errors failing the build (and ideally suppress the
noise).</p>
<p>These test all run on my local build server (under Jenkins and
Mstest) without any problem. (Other than the usual Selenium
flakiness :-( )</p>
<p>Thanks</p>
<p>eg From the console</p>
<p>1405954940886 addons.manager DEBUG Application has been
upgraded<br>
11081405954940913 addons.xpi DEBUG startup<br>
11091405954940915 addons.xpi DEBUG Skipping unavailable install
location app-system-local<br>
11101405954940915 addons.xpi DEBUG Skipping unavailable install
location app-system-share<br>
11111405954940922 addons.xpi DEBUG checkForChanges<br>
11121405954940937 addons.xpi DEBUG Directory state JSON differs:
cache [] state [{<a href=
"1401190050000}}}]">name":"app-global","addons":{"{972ce4c6-7e08-4474-a285-3208198ce6fd}":{"descriptor":"C:\Program
Files (x86)\Mozilla Firefox\browser\extensions\{9<br>
111372ce4c6-7e08-4474-a285-3208198ce6fd}","mtime":1402594368166,"rdfTime":1402020940000}}},{"name":"app-profile","addons":{"fxdriver@googlecode.com":{"descriptor":"C:\Users\appveyor\AppData\Local\Temp\anonymous.377c7c4fb3404453<br>
11148085a4870d1a58e2.webdriver-profile\extensions\fxdriver@googlecode.com","mtime":1405954940139,"rdfTime</a><br>
11151405954940952 addons.xpi-utils DEBUG Opening XPI database
C:\Users\appveyor\AppData\Local\Temp\anonymous.377c7c4fb34044538085a4870d1a58e2.webdriver-profile\extensions.json<br>
11161405954940953 addons.xpi DEBUG New add-on <a href=
"mailto:fxdriver@googlecode.com">fxdriver@googlecode.com</a>
installed in app-profile<br>
1117*** Blocklist::_loadBlocklistFromFile: blocklist is
disabled<br>
11181405954941425 addons.xpi-utils DEBUG Make addon
app-profile:<a href=
"mailto:fxdriver@googlecode.com">fxdriver@googlecode.com</a>
visible<br>
11191405954941427 DeferredSave.extensions.json DEBUG Save
changes<br>
11201405954941432 DeferredSave.extensions.json DEBUG Save
changes<br>
11211405954941432 addons.xpi DEBUG New add-on
{972ce4c6-7e08-4474-a285-3208198ce6fd} installed in app-global<br>
11221405954941435 addons.xpi-utils DEBUG Make addon
app-global:{972ce4c6-7e08-4474-a285-3208198ce6fd} visible<br>
11231405954941436 DeferredSave.extensions.json DEBUG Save
changes<br>
11241405954941436 DeferredSave.extensions.json DEBUG Save
changes<br>
11251405954941452 addons.xpi DEBUG Updating database with changes
to installed add-ons<br>
11261405954941452 addons.xpi-utils DEBUG Updating add-on states<br>
11271405954941453 addons.xpi-utils DEBUG Writing add-ons list<br>
11281405954942492 DeferredSave.extensions.json DEBUG Starting
timer<br>
11291405954942528 DeferredSave.extensions.json DEBUG Starting
write<br>
11301405954942727 DeferredSave.extensions.json DEBUG Write
succeeded<br>
11311405954942728 addons.xpi-utils DEBUG XPI Database saved,
setting schema version preference to 16<br>
11321405954942759 addons.manager DEBUG shutdown<br>
11331405954942760 addons.xpi DEBUG shutdown<br>
11341405954942765 addons.xpi-utils DEBUG shutdown<br>
11351405954942768 addons.xpi DEBUG Notifying XPI shutdown
observers<br>
11361405954942776 addons.manager DEBUG Async provider shutdown
done<br>
1137************************************************************<br>
1138* Call to xpconnect wrapped JSObject produced this error: *<br>
1139[Exception... "Component returned failure code: 0x8000ffff
(NS_ERROR_UNEXPECTED)
[nsIAnnotationService.getItemsWithAnnotation]" nsresult:
"0x8000ffff (NS_ERROR_UNEXPECTED)" location: "JS frame ::
<a>resource://app/components/nsBro</a><br>
1140wserGlue.js :: BG_EPDQI_runBatched :: line 1715" data: no]<br>
1141************************************************************<br>
1142System JS : ERROR
<a>resource://gre/modules/BookmarkHTMLUtils.jsm:897</a> -
NS_ERROR_UNEXPECTED: Component returned failure code: 0x8000ffff
(NS_ERROR_UNEXPECTED)
[nsINavBookmarksService.removeFolderChildren]<br>
1143System JS : ERROR
<a>resource://gre/modules/BookmarkHTMLUtils.jsm:897</a> -
NS_ERROR_UNEXPECTED: Component returned failure code: 0x8000ffff
(NS_ERROR_UNEXPECTED)
[nsINavBookmarksService.removeFolderChildren]<br>
1144<br>
11451405954943274 addons.xpi DEBUG startup<br>
11461405954943276 addons.xpi DEBUG Skipping unavailable install
location app-system-local<br>
11471405954943276 addons.xpi DEBUG Skipping unavailable install
location app-system-share<br>
11481405954943282 addons.xpi DEBUG checkForChanges<br>
11491405954943296 addons.xpi DEBUG No changes found<br>
1150*** Blocklist::_loadBlocklistFromFile: blocklist is
disabled<br>
1151!!! error running onStopped callback: TypeError: callback is
not a function</p></div>scascarinitag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T15:58:08Z2014-07-21T15:58:08ZSelenium Tests<div><p>I've tried to disable that logging for FireFox at no avail. I
suspect those messages come directly from FireFox to StdOut. Though
Chrome driver seems working the best.</p>
<p>Regarding "they're being flagged in red in the console" - could
you please drop a screenshot of console to see how it looks
like?</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T16:02:23Z2014-07-21T16:02:23ZSelenium Tests<div><p>Is this OK</p></div>scascarinitag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T16:04:18Z2014-07-21T16:04:18ZSelenium Tests<div><p>Not sure if that's too big - how about</p>
<p><a href=
"https://www.dropbox.com/s/flidvm5u78lr7pc/Screenshot%202014-07-21%2017.00.55.png">
https://www.dropbox.com/s/flidvm5u78lr7pc/Screenshot%202014-07-21%2...</a></p></div>scascarinitag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T16:05:51Z2014-07-21T16:05:51ZSelenium Tests<div><p>I see. Do you call <code>vstest.console</code> command as part
of your build script, not using tests "autodiscovery"?</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T16:07:47Z2014-07-21T16:07:47ZSelenium Tests<div><p>Yes because I want to run the tests after deployment.</p></div>scascarinitag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T16:12:37Z2014-07-21T16:12:37ZSelenium Tests<div><p>Perfect! The problem is that PowerShell treats everything
written to StdErr as exceptions. Switch "after deploy" script to
"CMD" mode - that should fix a problem.</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/338486202014-07-21T16:49:39Z2014-07-21T16:49:39ZSelenium Tests<div><p>OK tweaked that and kicked off a new build. All looking good so
far - tests are running and console is clear.</p>
<p>Thanks again.</p></div>scascarinitag:help.appveyor.com,2012-11-13:Comment/338486202014-07-22T10:35:24Z2014-07-22T10:35:24ZSelenium Tests<div><p>So the tests run as 'CMD' but run very slowly. About 5 ran in 30
minutes and then the build timed out.</p>
<p>Based on your comments above I did some experimenting and found
that if I set the <code>/InIsolation</code> flag I could switch
back to running under Powershell and I don't see any errors in the
console.</p>
<p>So I'm happy but you may want to investigate why the tests ran
so slowly under 'cmd'. I was running</p>
<p><code>vstest.console /logger:Appveyor
test/bin/Debug/Spiro.Angular.Selenium.Test.dll</code></p>
<p>as an after deployment script and it ran 5 selenium tests in 30
minutes</p>
<p>Currently I'm running</p>
<p><code>vstest.console /logger:Appveyor /InIsolation
test/bin/Debug/Spiro.Angular.Selenium.Test.dll</code></p>
<p>as Powershell and it runs about 50 selenium tests in 15 minutes
or so.</p>
<p>Thanks for all your help.</p></div>scascarinitag:help.appveyor.com,2012-11-13:Comment/338486202014-07-22T14:10:00Z2014-07-22T14:10:00ZSelenium Tests<div><p>Wow, that's really interesting observation! I will try to
understand this difference between PS and external command
mode.</p>
<p>Thank you for the update!</p></div>Feodor Fitsnertag:help.appveyor.com,2012-11-13:Comment/338486202014-11-27T12:05:14Z2014-12-25T07:50:25ZSelenium Tests<div><p>Hello guys,</p>
<p>Any updates on the issue? Why there is a time difference ?</p>
<p>We are also suffering from slow acceptance tests.</p></div>Rifat Erdem Sahintag:help.appveyor.com,2012-11-13:Comment/338486202014-11-27T19:02:00Z2015-04-03T15:33:57ZSelenium Tests<div><p>I believe it's been fixed for the new environment.</p>
<p>Look at that: <a href="https://ci.appveyor.com/project/FeodorFitsner/selenium-tests/build/1.0.10">
https://ci.appveyor.com/project/FeodorFitsner/selenium-tests/build/...</a>
- ran basic Selenium test against FF, Chrome and IE (!) drivers in
just 47 seconds. Artifacts contain screenshots.</p></div>Feodor Fitsner