Artifacts are not published after uploading ftp file

olegbevz's Avatar

olegbevz

27 Nov, 2017 10:36 PM

Hello,
I've recently added powershell script in "after tests script" section.
I use PSFTP module to upload file to the server:
$password = $password | ConvertTo-SecureString -asPlainText -Force
$credentials = New-Object System.Management.Automation.PSCredential($username,$password)
Set-FTPConnection -Credentials $credentials -Server $server -UsePassive -Session MySession -ignoreCert
Add-FTPItem -Path $path -LocalPath $localpath -Overwrite -Session MySession -BufferSize 100KB

Script executes successfully, but after that artifacts are not published due to an error:
Error uploading artifact the storage: The underlying connection was closed: An unexpected error occurred on a send.

Is it possible to get more detailed information about the error occured?

PSFTP link: https://gallery.technet.microsoft.com/scriptcenter/PowerShell-FTP-Client-db6fe0cb
Prject reference: https://ci.appveyor.com/projects

Thank you

  1. Support Staff 1 Posted by Ilya Finkelshte... on 28 Nov, 2017 04:17 AM

    Ilya Finkelshteyn's Avatar

    Hi Oleg,

    Project reference you sent is not exactly informative as it will show projects for user logged on, not your projects ;). For example for me it shows my projects.

    However I think I found you failed build. It is here, right?

    From what I see error happened when AppVeyor tries to upload build artifacts to the cloud storage, not when running your script. I have not enough enough information now, but it looks like SSL error. This is most probably your build specific issue, because otherwise we should have hundreds of complains now.

    Can you try to remove, or move to deployment script stage recently added command executed between this and this lines? I think they are now are being called as after test scripts and I believe (just speculation) that recently added command affect SSL settings on build VM, thus breaking artifacts upload. By removing or moving that recently added command (or whole script block) we can check this theory.

    Regarding FTP upload, this should work, but I am not sure build even reached that stage (because of artifacts upload issue). However I wonder why you do not consider built-in FTP support?

    Ilya.

  2. 2 Posted by olegbevz on 28 Nov, 2017 09:38 AM

    olegbevz's Avatar

    Hi Ilya,
    Thank you for response and sorry for incorrect link to appveyor projects.

    I've already had a lot of experiments commenting/uncommenting code lines in "after tests" script.
    Here are lines of code which breaks artifacts upload:
    Set-FTPConnection -Credentials $credentials -Server $server -UsePassive -Session MySession -ignoreCert
    Add-FTPItem -Path $path -LocalPath $localpath -Overwrite -Session MySession -BufferSize 100KB
    These lines of code is from FTP-Upload.ps1 script:
    https://bitbucket.org/olegbevz/justwatch/src/396b42205aadb430d23bf870dbceeec3e578a154/.appveyor/Ftp-Upload.ps1?at=features%2Fappveyor-export&fileviewer=file-view-default
    "Set-FTPConnection" and "Add-FTPItem" are Powershell Cmdlets from PSFTP package - one of the most popular PowerShell modules for FTP tasks.
    My assumption is that "after tests" script is executed in the same AppDomain as artifact publish logic and somehow breaks SSL settings.

    I haven't looked at FTP deployment provider yet. As understand I should publish file as artifact to deploy it to ftp server. But I don't want this file to be stored in build history.
    If PowerShell Cmdlets won't work perhaps I will migrate to FTP deployment provider...

  3. Support Staff 3 Posted by Ilya Finkelshte... on 29 Nov, 2017 07:48 AM

    Ilya Finkelshteyn's Avatar

    Can you run [Net.ServicePointManager]::SecurityProtocol before and after those commands and see if it changed?

  4. 4 Posted by olegbevz on 29 Nov, 2017 01:25 PM

    olegbevz's Avatar

    Ilya,
    I've updated the script: https://bitbucket.org/olegbevz/justwatch/src/e03ae16da0947910dd0a6a96fe17ae579b4a984d/.appveyor/Ftp-Upload.ps1?at=features%2Fappveyor-export&fileviewer=file-view-default
    Seems that before ftp upload following security protocols were specified on VM:Ssl3, Tls, Tls11, Tls12
    I tried to restored them after ftp upload but it didn't help.
    Perhaps there are more ssl settings which we need to restore...

  5. Support Staff 5 Posted by Ilya Finkelshte... on 30 Nov, 2017 05:02 AM

    Ilya Finkelshteyn's Avatar

    I am little bit lost as I cannot understand how it is being called from appveyor.yml. But anyway, does FTP upload works? Do you see files uploaded to your server?

  6. 6 Posted by olegbevz on 30 Nov, 2017 06:05 AM

    olegbevz's Avatar

    Hi Ilya, FTP upload works correctly, file is transfered to ftp server. The appveyor.yml from the root folder is not used by appveyor for some reason. In "after tests script" section I have the following line in script:
    ../.appveyor/Ftp-Upload -server $env:ftp_server -username $env:ftp_username -password $env:ftp_password -path html/yml/ -localpath list.xlsx
    Do you have an access to my project settings?

  7. Support Staff 7 Posted by Ilya Finkelshte... on 01 Dec, 2017 02:35 AM

    Ilya Finkelshteyn's Avatar

    AppVeyor does not respect YAML because your project is created as a generic Git, not BitBucket. It is explained here. I would recommend to re-create project as a BitBucket one.

    Do you really need to upload those files as artifacts as long as you already uploaded them to FTP? I mean we can root cause how that snap-in breaks SSL, but I am not sure you really need those artifacts.

  8. 8 Posted by olegbevz on 01 Dec, 2017 06:17 AM

    olegbevz's Avatar

    Ilya, thank you for the advice about Bitbucket repo. I will try it.
    I want to publish appveyor artifacts with binary files after uploading file to ftp. Its not the same artifacts. If the issue with SSL could not be solved easily I could try to create a separate appveyor project for ftp upload...

  9. Support Staff 9 Posted by Ilya Finkelshte... on 01 Dec, 2017 06:50 AM

    Ilya Finkelshteyn's Avatar

    We will solve it one way or another, no worries :) What if you upload to FTP as a deployment script? This happens after artifacts upload step.

  10. 10 Posted by olegbevz on 01 Dec, 2017 09:53 AM

    olegbevz's Avatar

    Ilya, I've made a workaround: I've moved ftp upload scripts to "On build success script" section. Now ftp upload executes after artifacts are published. But still it's only workaround: if you try to use PSFTP module it will break SSL settings in appveyor process and artifacts will not be published. Perhaps it would be better to execute powershell scripts in a separate app domain...
    Personally for me this solution is enough=)

  11. Support Staff 11 Posted by Ilya Finkelshte... on 05 Dec, 2017 01:04 AM

    Ilya Finkelshteyn's Avatar

    I agree. Can you try to wrap those commands as .ps1 script and call from cmd. Say - cmd: powershell.exe .\my-ftp-deploy.ps1. Let us know if it does not break artifact upload this way.

  12. 12 Posted by Mike Minns on 29 Dec, 2017 07:07 PM

    Mike Minns's Avatar

    Hi

    Just to say I seem to be having the same problem, artefacts are being found but can't be uploaded, see https://ci.appveyor.com/project/MikeCorsaro/sourcetreewin-85q5a/build/1.0.93

    I use an after_test section to run some scripts that create an installer of our project. The artefacts are based on the outcome of those scripts. As far as I can see those scripts complete successfully, the artefacts are found but the upload fails.

  13. Support Staff 13 Posted by Ilya Finkelshte... on 30 Dec, 2017 01:41 AM

    Ilya Finkelshteyn's Avatar

    Mike,

    I checked your build and I do not see that you use PSFTP which broke SSL settings in original post. Let us try to identify what happens in your case.

    Can you please insert [Net.ServicePointManager]::SecurityProtocol before and after your custom scripts (and any other steps you suspect can be the root cause) and see if some script/step changed that settings.

    If you found offending script, first try to add [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.SecurityProtocolType]::Ssl3 -bor [System.Net.SecurityProtocolType]::Tls -bor [System.Net.SecurityProtocolType]::Tls11 -bor [System.Net.SecurityProtocolType]::Tls12 after it.

    If this does not help, please share your build configuration in YAML format and offending script content (if you found it). Feel free to delete/replace all secrets from YAML and script.

    Ilya.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac