LeakyCLI: AWS and Google Cloud Command-Line Tools Can Expose Sensitive Credentials in Build Logs

In November last year, Microsoft fixed a severe security issue in Azure CLI that risked exposing credentials in logs. The vulnerability was identified as CVE-2023-36052 and given a CVSS score of 8.6. However, the Orca Research Pod recently discovered that the AWS and Google Cloud CLIs are exposed to the exact same vulnerability. Dubbed ‘LeakyCLI’, this is a vulnerability that can expose credentials in AWS and Google Cloud logs, which could have far reaching consequences.

Executive Summary:

• Orca has discovered that some commands on AWS CLI and Gcloud CLI can expose sensitive information, in the form of environment variables, which can be collected by adversaries when published by tools such as GitHub Actions.
• Microsoft faced the same issue in Azure CLI, and identified this vulnerability as CVE-2023-36052 (CVSS score of 8.6) and issued an update and recommendation. 
• If bad actors get their hands on these environment variables, this could potentially lead to view sensitive information including credentials, such as passwords, user names, and keys, which could allow them to access any resources that the repository owners can.
• CLI commands are by default assumed to be running in a secure environment, but coupled with CI/CD pipelines, they may pose a security threat.
• This bypasses secret labeling, which aims to block sensitive exposure, because the credentials that are printed back to stdout were never defined by the user during the automation setup.
• Upon discovery of the vulnerability, Orca informed both Google and AWS, who responded that they consider this to be expected behavior based on current design. 
• To prevent exposure to this AWS and Google Cloud CLI vulnerability, organizations are advised to avoid storing secrets in environment variables, and instead retrieve them from a dedicated secrets store service such as AWS Secrets Manager.

What are Azure, Gcloud and AWS CLI?

All three major cloud service providers provide command-line interfaces for interacting with their cloud platforms: Azure CLI, AWS CLI, and Gcloud CLI The CLIs are unified tools to manage cloud services, which transparently send Rest API requests via documented commands.

These CLIs are most commonly used in a local private environment, like a developer’s personal computer, but they can also be used for Continuous Integration and Continuous Deployment (CI/CD) environments. A simple example of a CI/CD use case would be deploying source-code to a Lambda function every time there’s a push event to master.

name: AWS CI

on:
  push:
    branches:
      - master

jobs:
  deploy:
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
      - name: Set up AWS CLI
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ secrets.AWS_REGION }}
      - name: deploy
        run: |
          npx ncc build src/index.js
          zip -j deploy.zip ./dist/*
          aws lambda update-function-code --function-name MyFunction --zip-file fileb://deploy.zip

Exposure of Serverless environment variables

The main problem we observed was in serverless such as Azure Functions (however this issue was fixed by Microsoft as mentioned above), Google Cloud Functions and AWS Lambda.The documented APIs for these services include actions that return the configuration for these resources, including their environment variables.

What I found interesting is that it is not only the get or describe commands that return the configuration (including environment variables), but also update and delete. Which carried significant misconfigurations out in the wild.

AWS CLI Leakage

aws lambda get-function-configuration
aws lambda get-function
aws lambda update-function-configuration
aws lambda update-function-code
aws lambda publish-version

In a similar way, the above commands send existing environment variables back to the stdout, even if they weren’t part of the associated command.

Gcloud CLI Leakage

gcloud functions deploy <func> --set-env-vars
gcloud functions deploy <func> --update-env-vars
gcloud functions deploy <func> --remove-env-vars

The above commands send the defined / predefined environment variables back to stdout. Or in the advanced scenario, back to the build logs. If the developer isn’t aware of it, even using secret masking via GitHub Actions / Cloudbuild will not do, because there may be pre-existing environment variables in the cloud function.

Exploitation Proof of Concept

AWS

On Github search, we ran the following command, targeting a leak via Github Actions, CircleCI and TravisCI, which resulted with over 1k hits.

"aws lambda" AND ("update-function-configuration" OR "update-function-code" OR "publish-version") AND (path:.github/workflows OR path:.circleci OR path:.travis)

Then, the procedure became manual and thus limited. We entered a few repositories and looked at their build logs. We encountered many dozens of projects that inadvertently leak information that could be considered sensitive in environment variables, including passwords and keys. Whilst some AWS-specific parameters like account ID’s or S3 bucket names aren’t considered private, customers could unknowingly expose values of their own using this mechanism.leaked sensitive information, ranging from account IDs to IAM roles and all the way to passwords and keys.

Real-life examples

GCP

On Github search, we ran the following command, targeting a leak via Github Actions, CircleCI, TravisCI and this time also Cloudbuild, which resulted with only 137 hits.

"gcloud functions deploy" AND ("--set-env-vars" OR "--update-env-vars" OR "--remove-env-vars") AND (path:.github/workflows OR path:.circleci OR path:.travis OR path:cloudbuild)

Again, we were able to find leaked sensitive data in the form of project names, service accounts and environment variables.

If you wonder how we could fetch cloudbuild logs without access to the associated project, GCP provides a cloudbuild GitHub integration through the cloudbuild Github app and allows for build logs to be viewed inside GitHub on an opt-in, per-build-trigger basis

Since we couldn’t find many interesting exploitation examples this time, we decided to demonstrate a local scenario. The fact that we indeed found examples validates the use case, which can pose an internal information disclosure threat, which could lead to a local Privilege Escalation.

Cloud Build Logs Viewer

It seems like there is no granular GCP role to allow viewing the cloud build logs (except for Viewer), even when it is a highly requested feature by users. The workaround that is used by many users is to forward the cloud build logs to a cloud storage bucket, and then provide the required users with access to this bucket.

Another workaround is storing the logs in cloud logging, where it will be visible to users with Logs Viewer permissions according to this document.

Local Privilege Escalation

It’s logical to allow the creator of a build to view its build logs. The problem begins when the permission system gets corrupted due-to a complicated usage. Providing read access to Cloud Storage or Cloud Logging shouldn’t escalate to other services. Something that can indeed be manifested with the exposure of leaked environment variables.

Vendor Response

AWS

AWS considers this to be expected behavior based on the current design and documentation available to customers. Upon the examples we have provided and since they understand the risk, they have updated their documentation to better assist customers.

AWS outlines the following recommendations:

1. Do not use environment variables to store sensitive values of your serverless resources. Instead have your serverless code programmatically retrieve the secret from your CSPs secrets store (e.g AWS Secrets Manager).
2. Review the contents of your build logs to ensure they do not contain sensitive information within them. Consider approaches such as piping to /dev/null to suppress command outputs. 
3. Consider the access of your logs and scope access appropriately for your use case.

GCP 

GCP considers this to be expected behavior. They emphasize that output can be suppressed by using the `–no-user-output-enabled` flag. They also recommend using Secrets manager functions built into gcloud deploy command line to store credentials. These are the `–set-secrets` and `–update-secrets` options in the documentation.

Orca Recommendations

We strongly advise organizations to follow the vendors’ recommendations above. Since this issue can still be present, it is highly important to follow the best practices:

1. Having secrets inside environment variables of serverless resources isn’t recommended. Instead, users should define sensitive variables in the associated managed Secrets Manager service and retrieve them from the serverless code.
2. Make sure to pipe any command outputs that aren’t required inside the build logs to /dev/null. This will prevent unnecessary exposure of sensitive information.

Get a Demo of the Orca Cloud Security Platform

To see how Orca can help you secure your cloud environments on AWS, Azure, Google Cloud, Oracle Cloud, and Alibaba Cloud, as well as mitigate and investigate any vulnerabilities, schedule a 1:1 demo with one of our experts.