Deploy to S3 with YAML and secure setting

YakDriver's Avatar

YakDriver

07 Jun, 2018 07:24 PM

What I Want to Happen

I want to deploy to S3 using appveyor.yml, and provide the access key ID and secret key through Project -> Settings -> Environment -> Environment Variables ("secure settings").

I prefer not to use: 1. an account-level Environment (since an Environment moves important implementation details out of appveyor.yml, spreading maintenance); 2. a project-level Settings -> Deployment (which makes important implementation details invisible in appveyor.yml); and 3. appveyor.yml directed assignment of environment variables, like access_key_id: $(AWS_ACCESS_KEY_ID) (because it puts unnecessary implementation details in appveyor.yml).

Generally, integrations with the AWS SDK work properly when AWS_ACCESS_KEY_ID AND AWS_SECRET_ACCESS_KEY are set in the environment. AppVeyor should follow this convention.

Here is the S3 portion of appveyor.yml I'd like to use (in it's entirety):

  - provider: S3
    bucket: $(DEV_BUCKET)
    folder: $(S3_KEYFIX)
    on:
      branch: develop
I'd like to define AWS_ACCESS_KEY_ID in the secure settings for the project and then have the S3 deployment provider use that variable from the environment.

What Actually Happens

I've tried using AWS_ACCESS_KEY_ID, ACCESS_KEY_ID and access_key_id as secure settings and always get the same error:

Deploying using S3 provider
Provider setting not found or it's value is empty. If secure setting is used please check that value was encrypted (or YAML was exported) while being logged under correct account.
Parameter name: access_key_id
dir "env:"
How can this be accomplished?
  1. Support Staff 1 Posted by Ilya Finkelshte... on 07 Jun, 2018 09:52 PM

    Ilya Finkelshteyn's Avatar

    As I understand you want to keep sensitive variables in UI, but use them in YAML. This is possible, but you still need to explicitly set them in S3 deployment settings like this:

    - provider: S3
      access_key_id: $(AWS_ACCESS_KEY_ID)
      secret_access_key: $(AWS_SECRET_ACCESS_KEY)
      bucket: $(DEV_BUCKET)
      folder: $(S3_KEYFIX)
    

    Sure, variables names can be any, not exactly AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

    If you want to completely hide everything sensitive (even in encrypted form) from YAML, you can use environment deployment. However be aware that this kind of deployment is asynchronous and failed deployment will not fail the build. More details are here. You can always set separate notification fro environment deployment though.

    Please let us know if this makes sense and if it fits your scenario.

  2. 2 Posted by YakDriver on 08 Jun, 2018 12:14 PM

    YakDriver's Avatar

    Explicitly setting the access key id and secret access key in S3 deployment YAML does work for me, but it seems like it should be unnecessary when the AWS SDK will use the proper environment variables if allowed.

    For example, in Travis CI, setting AWS_ACCESS_KEY_ID in the UI as secure, the S3 provider will then implicitly use that environment variable without anything being set in the YAML.

    It's valid in .travis.yml, if AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are set in the UI, to just skip these provider parameters:

    - provider: s3
      bucket: $DEV_BUCKET
      upload-dir: $S3_KEYFIX
      skip_cleanup: true
      on:
        branch: develop
    

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac

 

18 Aug, 2018 11:54 AM
18 Aug, 2018 12:13 AM
17 Aug, 2018 06:02 PM
17 Aug, 2018 04:26 PM
17 Aug, 2018 04:04 PM
17 Aug, 2018 06:31 AM