I like the idea of being able to use parameter files. The git-sync format is probably the best format as it associates the parameters and the template file to be able to do this work. I think second would be able to support the AWS CLI parameter key/value file.

It would really cut down on alternative if/then/else processing we do. I would love to efficiently do this so we aren't doing duplicate processing of things that aren't changing.

I think it would also be beneficial to support the configuration json file used by codepipeline. It would involve some kind of mapping file as the codepipeline stage declarations tells cloud formation which template and parameter file to use for each stack. In my environment I'll use the same cf template with many jsons in the same pipeline.

Allow cfn-lint to use deployment files to lint and fill in parameters

• Original Author(s):: @kddejong

Summary

Allow cfn-lint to use deployment files instead of directly providing templates. Deployment files will contain paths to template and parameter files. cfn-lint will use these files to validate different templates and parameter scenarios.

Example

Given the template:

Parameters:
  Environment:
    Type: String
    AllowedValues:
    - production
    - stage
    - development
  BucketName:
    Type: String
Resources:
  Bucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref BucketName
      Tags:
      - Key: Environment
        Value: !Ref Environment

and a deployment file production.yaml

template-file-path: template.yaml
parameters:
  Environment: production
  BucketName: foo*bar

and a deployment file development.yaml

template-file-path: template.yaml
parameters:
  Environment: dev
  WrongParameter: my-bucket

We would get errors like:

EXXXX 'foo*bar' does not match '^[a-z0-9][a-z0-9.-]*[a-z0-9]$' when resolved by deployment file 'production.yaml'
template.yaml:14:7
EXXXX 'dev' is not one of ['production', 'stage', 'development']
development.yaml:3:2
EXXXX 'WrongParameter' is not one of ['Environment', 'BucketName']
development.yaml:4:2
EXXXX 'BucketName' is a required property
development.yaml:2:1

Motivation

Currently, cfn-lint validates Default and AllowedValues for parameters, which can result in validating more scenarios than intended. Using deployment files would:

• Limit the number of validation scenarios
• Validate parameter value correctness
• Verify parameter existence in templates

Details

• Add a new cfn-lint configuration parameter called --deployment-files which can take a glob compatible string to any number of deployment files. This parameter is exclusive with templates as this may cause confusion with the runner.
• Expand the runner capabilities to include deployment files.
• Create a pluggable system to handle deployment files and create the initial plugin for Git sync deployment files.
• Process the template replacing Ref to parameters with their appropriate value
• Process any functions that can be fully resolved. If they can't be fully resolved we leave them as they are.
• Run rules against the resolved template
  • Note if we process the templates replacing parameters for Ref values certain rules will have to be disabled. Example I3042, W3010, W8003, etc.

Limitation

• Resolution of Pseudo-Parameters may be limited depending on the information in the deployment file.
  • AWS::AccountId, AWS::Region, AWS::Partition, AWS::StackId, AWS::StackName, AWS::NotificationArns, AWS::URLSuffix

Outstanding questions

• While cfn-lint has made great strides for how it handles parameter combinations this may not be efficient. Ideally we would only change how a template works as we run into a parameter value however this can be complicated. It may be easier to process a template with the parameter values then lint the processed template.
• Have to create a mechanism to validate the correctness of deployment files. One method is to create a json schema document based on the template then use that schema to validate the deployment file. From the example this will cover any of the errors that we display in the deployment files themselves.

References

• CloudFormation Git sync
• CodePipeline CloudFormation action
• GitHub CloudFormation deploy