Home → Suggestions →

Intel Parallel Studio

• Edit

robertpancoast77

20 Jan, 2017 12:38 AM

Greetings,
1.) imagine a "chocolatey package" to quickly perform silent install of the Intel Stack in Windows CI Server.

2.) We need to be able to specify the Intel Stack Registration Information using a secured environment variable.

REF: https://software.intel.com/en-us/forums/intel-visual-fortran-compiler-for-windows/topic/628773#comment-1895874
Thanks,
Rob

1. 1 Posted by Ilya Finkelshte... on 20 Jan, 2017 03:05 AM

   

   Hi Robert,

   From what I understand from link you sent, your CI process requires Intel® Manycore Platform Software Stack (Intel® MPSS) to be installed on build machine.

   AppVeyor instantiates fresh VM for every build, and standard AppVeyor VHD does not contain this software. Therefore you need to install it as part of build scenario. AppVeyor build pipeline has install stage where you can add script which silently installs this software.

   This can be chocolatey package (if somebody wrote it), can be just script which installs this silently. I tried and compiled something like this:

   $zipPath = "$($env:USERPROFILE)\mpss-3.8.1-windows.zip"
   (New-Object Net.WebClient).DownloadFile('http://registrationcenter-download.intel.com/akdlm/irc_nas/11193/mpss-3.8.1-windows.zip', $zipPath)
   7z x $zipPath -y -ompss | Out-Null
   & '.\mpss\mpss-3.8.1\Intel(R) Xeon Phi(TM) coprocessor essentials.exe' /S /v/qn
   sleep 60 #sleep to ensure installation completed
   

   What about secure variables, you can use them for your secrets, more information here

   Thanks!
   Ilya.

   • Edit

2. 2 Posted by robertpancoast7... on 20 Jan, 2017 05:17 PM

   

   Greetings Ilya,

   We require both mpss and Intel Parallel Studio 2017 installed in Appveyor VM. This enables compilation for native k1om architecture using HPC Intel stack.

   reference:
   https://github.com/dotnet/corert/issues/2519

   Thanks,
   Rob

   • Edit

3. 3 Posted by Ilya Finkelshte... on 20 Jan, 2017 10:54 PM

   

   Hi Robert,

   I googled this article. I believe you can figure out how to write silent install script from here, having previous sample and that article. I recommend to debug script on local machine, and after that try it as part of appveyor build.

   Some notes which may help you:

   • Here you indeed need to use secure variables for serial number.

   • I see that download package itself is not interactive. You may need to download it manually once and the store in the build cache

   Hope this helps. Please let us know if you have any specific issues doing it.

   Ilya.

   • Edit

4. 4 Posted by robertpancoast7... on 20 Jan, 2017 11:48 PM

   

   Greetings Ilya,

   Let's assume we use a powershell entry for install: during pipeline.
   Given the Intel package is install in ./intel-stack and the install size is 9GB, can appveyor cache support a cache directory for 9Gb of state of art Intel compiler?

   What is syntax for cache functionality given automatic download/install/activation? We need to provide maximum time for icpc and icc to compile the target project and only down the huge file once.

   Thanks,
   Rob

   • Edit

5. 5 Posted by robertpancoast7... on 21 Jan, 2017 07:00 PM

   

   I have attempted the "build-cache" instructions in the past and failed: Updating build cache...
   Cache 'C:\Users\appveyor\AppData\Local\NuGet\Cache' - Uploading (8,769,957 bytes)...100%Error uploading cache entry to the cache storage: Remote server returned 500: The stream does not support concurrent IO read or write operations.

   REFERENCE:
   Line #9939
   https://ci.appveyor.com/project/buddhi1980/mandelbulber2/build/1.0.1.0.0.7.16-master/job/hjtjgtkukr85vc03

   • Edit

6. 6 Posted by Ilya Finkelshte... on 21 Jan, 2017 08:44 PM

   

   Hi Robert,

   This is because you are over build cache size limit. Sorry for unfriendly error, we have bug on it. Cache size limits are described here.

   However with this big file size to download, I don't think it is best approach to download, unzip and install every time. I mean even if you upgrade to premium plan, your build will be too long with those massive install steps.

   What maybe more suitable for you is "private build cloud" feature we are releasing now. Idea is to let AppVeyor to manage all logic, but have your own VM infrastructure. Thus you can maintain your own VHD image with all you need pre-installed. We support different types of clouds, but documentation is ready for Azure only now, please check here.

   Please let us know what direction you decide to go, and if you have any additional questions.

   Ilya.

   • Edit

7. 7 Posted by robertpancoast7... on 22 Jan, 2017 04:26 PM

   

   Greetings,

   I request "private build cloud" feature for azure.
   Does this option require a paid Azure subscription?
   I envision cross-compilation on the cloud forall relevant k1om libraries and applications.
   This requires many resources, but all dependencies for mandelbulber project fall well within open-source domain.

   I wish to continue this discussion on Github.
   REFERENCE:
   https://github.com/appveyor/ci/issues/1276

   Thanks for helpful information!

   • Edit

8. 8 Posted by Ilya Finkelshte... on 23 Jan, 2017 06:18 PM

   

   OK, answered on GitHub, lets close this thread then and continue on GitHub.

   • Edit

9. Ilya Finkelshteyn closed this discussion on 23 Jan, 2017 06:18 PM.