atlassian/convert-helm-values repository overview

⁠Convert Helm Values

** For Atlassian Internal Use Only **

This pipe converts archetype descriptor or a service descriptor to KITT Helm values.

⁠Configuration File

The CONFIG variable should contain the conversion configuration in YAML format. The configuration can include the following fields:

• kushi: [required] all available arguments for kushi migrate command, you need figure out by run atlas kushi migrate --help locally
  • archetype-descriptor: [required] the path to the archetype descriptor file
  • output-dir: [required] the output directory for the Helm charts
  • ... other kushi arguments as needed
• values-overrides: [optional] refer to values-overrides Schema, a list of values overrides to be applied to the Helm charts
• values-removals: [optional] refer to values-removals Schema, a list of values fields to be removed from Helm values
• archetype-removals: [optional] [monolith only] a list of archetype fields to be removed when converting SD to Archetype descriptor
• archetype-overrides: [optional] [monolith only] archetype fields to be overridden when converting SD to Archetype descriptor

⁠values-overrides Schema

• file: [required] the path to the values overrides file, relative to kushi.output-dir e.g. values.yaml, stg-east/values.yaml
• values: [required] a map of values to be overridden in the Helm charts, e.g. {"image.tag": "latest"}

⁠values-removals Schema

• file: [required] the path to the values removals file, relative to kushi.output-dir e.g. values.yaml, stg-east/values.yaml
• values: [required] a list of values fields to be removed from the Helm charts.

Examples,

values-overrides:
  - file: values.yaml
    values:
      kitt-application:
        secrets:
          vault: { }
  - file: stg-east/values.yaml
    values:
      kitt-application:
        secrets:
          vault:
            static:
              shard-purpose: ""
        listFields: [ "a" ]

⁠Example

kushi:
  service: confluence-dev-us-11
  environment: stg-east,649535320606
  archetype-descriptor: ./tests/fixtures/archetype-descriptor.yaml
  output-dir: ./helm/confluence-dev-us-11
  main-container: confluence
  request-cpu: 5000m
  request-memory: 16Gi
  limit-cpu: 5000m
  limit-memory: 16Gi
  worker:
    - AmqWorker,5000m,16Gi,5000m,16Gi
  sidecar:
    - cc-nginx-sidecar,500m,100Mi,500m,350Mi
    - hofund,500m,100Mi,500m,350Mi
  ignore-error:
    - multiple-load-balancers
    - sidecar-poco
    - unsupported-characters
  debug: "" # set an options with empty string to use it as a flag, similar to --debug in CLI
  skip-docker-distributor-generation: ""
  skip-kitt-descriptor-generation: ""
  values-overrides: config/values-overrides.yaml # we could use a file to override values, but it's not flexible enough and cannot completely remove a field, that's why we have this pipe

values-removals:
  - file: values.yaml
    values:
      kitt-application:
        secrets:
          vault:
            static:
              shard-purpose: { } # remove the field completely

values-overrides:
  - file: blah/values.yaml # it will skip, if the file does not exist
    values: { } # it does nothing, if no values are provided
  - file: stg-east/values.yaml
    values:
      kitt-application:
        global:
          kitt:
            v1alpha2:
              cluster:
                cloud_type: gcp # add the field if does not exist, or modify the value if it differs
  - file: values.yaml # it will override the values in the file
    values: { }

archetype-removals:
  definitions:
    serviceDescriptor:
      definitions:
        isolatedCloudServiceDescriptor: { } # ic conversion throws too many errors, so we need to remove it completely for now

archetype-overrides:
  definitions:
    serviceDescriptor:
      compose:
        upp-sidecar:
          ports:
            - 8181:8484 # origin was 8181:8181 which conflicts with poco-sidecar

For the above example, for values.yaml, it will completely remove the kitt-application.secrets.vault field, and for stg-east/values.yaml, it will remove the kitt-application.secrets.vault.static.shard-purpose field. In addition, it will remove the kitt-application.listFields.a field from the list.

Please be noted,

• If the value is a list, it will remove any item in the list that matches the value.
• To remove the field completely, the value has to be an empty string, empty object or empty list, otherwise it will not be removed.

⁠Pipe

⁠Workflow

1. The pipe reads the CONFIG variable, which contains the conversion configuration.
2. If the SERVICE_DESCRIPTOR variable is provided, it will convert the service descriptor to an archetype descriptor.
3. The pipe will then call atlas kushi migrate command to convert the archetype descriptor to Helm charts.
4. Then the pipe read config overrides from the configuration file and apply them to the Helm values.
5. If GIT_ENABLED is set to true, it will commit the changes to the git repository.
6. If PULL_REQUEST_ENABLED is set to true, it will create a pull request with the changes.

⁠Variables

⁠Artifactory Authentication

The artifactory credentials are expected and will be used to log into Helm repository to generate Helm charts.

This can be obtained by using the atlassian/artifactory-sidekick:v1 pipe, the pipe will export the following environment variables:

• ARTIFACTORY_USERNAME
• ARTIFACTORY_PASSWORD

⁠Bitbucket Access Token

To use Bitbucket for git operations, you need to provide a repository access token with the required permissions. This can be set in the GIT_PASSWORD variable.

1. Go to Repository settings in Bitbucket.
2. Navigate to Access tokens in the left sidebar.
3. Click on Create access token, the token name will be the Bot name, so choose a meaningful name.
4. Set the expiration date as needed, maximum 1 year.
5. Create an access token with the following permissions:
   • Repositories: Read, Write
   • Pull requests: Read, Write
   • Pipeline: Read
6. Copy the generated token and set it in the GIT_PASSWORD variable.
7. Copy the generate email as GIT_EMAIL variable.
8. Copy the username (which will be x-token-auth) as GIT_USERNAME variable.

Please be noted that the username is always x-token-auth for Bitbucket, therefore you could rely on the default value for GIT_USERNAME.

Once you have obtained the access token, you can set it in the GIT_PASSWORD pipeline variable.

1. Go to Repository settings in Bitbucket.
2. Navigate to Repository variables in the left sidebar.
3. Add a new variable with the name GIT_PASSWORD and paste the access token (secret).
4. Add a new variable with the name GIT_EMAIL and paste the email address you used to create the access token.

⁠Bitbucket Pipeline Setup

In your bitbucket-pipelines.yml file, you can use the pipe as follows:

pipelines:
  custom:
    convert-helm-values:
      - step:
          name: Convert Helm Values
          script:
            - pipe: atlassian/artifactory-sidekick:v1 # must have this to prepare for artifactory authentication
            - source .artifactory/activate.sh
            - pipe: docker://atlassian/convert-helm-values:latest
              variables:
                CONFIG: 'convert-helm-values.yaml'
                SERVICE_DESCRIPTOR: 'path/to/service-descriptor.yaml' # optional, ONLY FOR MONOLITH, if you want to convert SD to Archetype descriptor first
                ARTIFACTORY_USERNAME: ${ARTIFACTORY_USERNAME} # required for Helm repository authentication, exported by artifactory-sidekick
                ARTIFACTORY_PASSWORD: ${ARTIFACTORY_PASSWORD} # required for Helm repository authentication, exported by artifactory-sidekick
                DEBUG_MODE: 'true' # optional, set to true to enable debug mode
                GIT_ENABLED: 'true' # optional, set to true to enable git operations
                GIT_EMAIL: ${GIT_EMAIL} # optional, set the git email address in pipeline variables
                GIT_USERNAME: ${GIT_USERNAME} # optional, set the git username in pipeline variables
                GIT_PASSWORD: ${GIT_PASSWORD} # optional, set the git access token in pipeline variables
                PULL_REQUEST_ENABLED: 'true' # optional, set to true to enable pull request creation