Home → Problems →

Deploy to S3 with YAML and secure setting

• Edit

YakDriver

07 Jun, 2018 07:24 PM

What I Want to Happen

I want to deploy to S3 using appveyor.yml, and provide the access key ID and secret key through Project -> Settings -> Environment -> Environment Variables ("secure settings").

I prefer not to use: 1. an account-level Environment (since an Environment moves important implementation details out of appveyor.yml, spreading maintenance); 2. a project-level Settings -> Deployment (which makes important implementation details invisible in appveyor.yml); and 3. appveyor.yml directed assignment of environment variables, like access_key_id: $(AWS_ACCESS_KEY_ID) (because it puts unnecessary implementation details in appveyor.yml).

Generally, integrations with the AWS SDK work properly when AWS_ACCESS_KEY_ID AND AWS_SECRET_ACCESS_KEY are set in the environment. AppVeyor should follow this convention.

Here is the S3 portion of appveyor.yml I'd like to use (in it's entirety):

  - provider: S3
    bucket: $(DEV_BUCKET)
    folder: $(S3_KEYFIX)
    on:
      branch: develop

I'd like to define AWS_ACCESS_KEY_ID in the secure settings for the project and then have the S3 deployment provider use that variable from the environment.

What Actually Happens

I've tried using AWS_ACCESS_KEY_ID, ACCESS_KEY_ID and access_key_id as secure settings and always get the same error:

Deploying using S3 provider
Provider setting not found or it's value is empty. If secure setting is used please check that value was encrypted (or YAML was exported) while being logged under correct account.
Parameter name: access_key_id
dir "env:"

How can this be accomplished?

1. 1 Posted by Ilya Finkelshte... on 07 Jun, 2018 09:52 PM

   

   As I understand you want to keep sensitive variables in UI, but use them in YAML. This is possible, but you still need to explicitly set them in S3 deployment settings like this:

   - provider: S3
     access_key_id: $(AWS_ACCESS_KEY_ID)
     secret_access_key: $(AWS_SECRET_ACCESS_KEY)
     bucket: $(DEV_BUCKET)
     folder: $(S3_KEYFIX)
   

   Sure, variables names can be any, not exactly AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

   If you want to completely hide everything sensitive (even in encrypted form) from YAML, you can use environment deployment. However be aware that this kind of deployment is asynchronous and failed deployment will not fail the build. More details are here. You can always set separate notification fro environment deployment though.

   Please let us know if this makes sense and if it fits your scenario.

   • Edit

2. 2 Posted by YakDriver on 08 Jun, 2018 12:14 PM

   

   Explicitly setting the access key id and secret access key in S3 deployment YAML does work for me, but it seems like it should be unnecessary when the AWS SDK will use the proper environment variables if allowed.

   For example, in Travis CI, setting AWS_ACCESS_KEY_ID in the UI as secure, the S3 provider will then implicitly use that environment variable without anything being set in the YAML.

   It's valid in .travis.yml, if AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are set in the UI, to just skip these provider parameters:
   

   - provider: s3
     bucket: $DEV_BUCKET
     upload-dir: $S3_KEYFIX
     skip_cleanup: true
     on:
       branch: develop
   

   • Edit

3. Ilya Finkelshteyn closed this discussion on 25 Aug, 2018 02:48 AM.