Sensitive Data Discovery: Tools, Best Practices & How It Works

Key Takeaways

Sensitive data discovery locates the stores holding regulated or confidential data across a cloud estate, including the ones nobody recorded when they were created.
Every discovery product samples instead of reading everything. The honest question about any tool is what share of the estate it can reach.
Four categories of tooling compete for this job, and each was built around a different collection model with a different blind spot.
GDPR Article 30 and PCI DSS Requirement 12.5.2 both turn the data inventory into a documented obligation with a stated review cadence.
Orca discovers data stores across cloud accounts without an agent on each workload, then attaches exposure, identity reach, and attack path to every finding.

Sensitive data discovery finds where regulated, confidential, and personal data sits across an estate, then records each location so a security team can act on it. It runs against object storage, managed databases, snapshots, backups, and analytics warehouses. The practice exists because a hand-drawn map of where data lives falls out of date the moment a team creates a store that is not on it.

The stores that cause incidents are rarely the ones on the diagram. A decommissioned analytics bucket still holds a production export from the migration that replaced it. A database snapshot gets copied into a sandbox account for a load test, and the copy outlives the test. Both were created for a legitimate reason, and neither was inventoried, so neither is covered by the controls protecting the original.

This guide covers what discovery is and how the software finds data. It then maps the tool categories, shows where each one breaks, and ends on what to do with a finding once you have one.

What is Sensitive Data Discovery?

Sensitive data discovery is the scanning and profiling process that identifies which data stores hold sensitive content. It reports each one as a finding tied to a specific resource. The output is a location, not a verdict about a single record. A discovery run answers where to look, and it answers repeatedly, because the estate keeps changing underneath the answer.

Scope is what separates this from a database audit. A cloud estate holds structured data in managed databases and warehouses, semi-structured data in exports and logs, and unstructured files in object storage. It also holds copies: snapshots, replicas, backups, and the artifacts of every migration that ever ran. The wider practice of data security posture management treats discovery as its first component, then builds posture and access analysis on top of it.

Discovery, Inventory, and Classification Are Three Different Jobs

The three words often get used for one activity, which is where a discovery project starts to drift. Discovery finds candidate stores and reports what they appear to contain. Inventory records each store as a durable entry with an owner, a purpose, a retention rule, and a jurisdiction. Classification decides what the contents are.

The three fail in different ways. Discovery fails by missing a store. Inventory fails when nobody updates the entry after the store changes shape, so an auditor reads a document describing last year’s environment. A team that runs a scan and files the CSV has done the first job and skipped the second.

How Sensitive Data Discovery Software Works

Sensitive data discovery software runs a repeating loop. It enumerates the resources in an account, obtains read access, and pulls a sample of the contents. It then inspects that sample for recognizable data types and writes the result back as a profile on the resource. Nothing in that loop reads every byte an organization stores, because sampling is the design and the provider documentation says so plainly.

The Scan Pipeline

Amazon’s automated sensitive data discovery in Macie states the mechanism plainly. The service “continually evaluates your S3 bucket inventory and uses sampling techniques to identify and select representative S3 objects in your buckets.” It then analyzes those objects and produces both findings and per-object discovery results.

Microsoft calls the same idea smart sampling, and its data security posture management documentation gives the numbers. For cloud storage, the scanner “samples up to 20 MB for each file scanned within the container.” For databases, it “selects certain tables and samples between 300 to 1,024 rows using nonblocking queries.”

Google structures the output instead. Sensitive Data Protection produces data profiles at the project, table, and column level, each carrying predicted data types alongside calculated risk and sensitivity levels. A profile is an assertion about a resource that a policy engine can evaluate. A finding is one observation inside it.

Connected, Agent Based, and Out of Band Collection

How a scanner reaches the data decides what it can never see. Three collection models are in use, and most platforms combine two of them.

Connected and API based collection authenticates to a service and reads through its own interface. It needs credentials and a resource the API lists. Google’s discovery service requires connection details for each Cloud SQL instance before profiling starts. It also names resources it cannot profile at all, including BigQuery Omni tables and non-BigLake external tables such as Google Sheets. An unlisted resource in an unmonitored account stays invisible.
Agent based collection installs software on the host and reads from inside it. That view reaches local files and process activity, and it stops where the agent was never installed. The gaps come from deployment: a managed service that accepts no agent, or an account outside the deployment pipeline.
Out of band collection reads storage separately from the running workload, using snapshots or block-level copies. It touches no production process and needs no per-workload install, so it scales across accounts. It sees what reaches storage, so data that only exists in memory is beyond it, and a store created after the last cycle waits for the next one.

None of the three is a virtue on its own. Ask which resources each model can enumerate in your accounts, and which ones fall in the gap between them.

Sensitive Data Discovery and Classification

Sensitive data discovery and classification are separate steps that only pay off together. Discovery says a store exists and holds something that looks regulated. Classification says what the contents are: cardholder data, health records, employee files, or a synthetic test set resembling all three. An inventory without labels cannot drive a control, since a policy blocking public access to health records has to know which buckets hold them.

The chain runs one way. A store that discovery never found cannot be classified, so classification accuracy is bounded by discovery coverage. Microsoft’s documentation marks that boundary: its sensitive data discovery “detects sensitive data at the cloud workload level” and “might not detect all types.” How a classifier decides what a piece of data contains, and how its output should be read, belongs to the classification side of the pipeline.

Benefits of Automated Sensitive Data Discovery

A pattern detector looks for a character sequence, validates it, and checks what sits nearby. Microsoft’s sensitive information types build each definition from four parts: a primary element, supporting elements, a confidence level, and a proximity window. The primary element can be a regular expression with or without a checksum, a keyword list, a keyword dictionary, or a function.

Supporting elements are where the score comes from. Microsoft’s example requires a nine-digit pattern to appear within 250 characters of an account number, a name, or a date of birth. Confidence follows the evidence count, at fixed values of 65 for low, 75 for medium, and 85 for high. On well-formed identifiers with a checksum, this approach is the most predictable of the three, because a checksum either validates or it does not.

Supervised Classifiers

Automated sensitive data discovery earns its place by running on a schedule the estate cannot outpace. A quarterly manual review describes an environment that has already changed. Four benefits carry the budget line.

New stores enter scope without a ticket. The sandbox copy of a production snapshot gets profiled on the next cycle instead of at the next audit.
Findings carry a resource, not a filename. You get an account, a region, a service, and an owner tag, which is what a remediation ticket needs.
Evidence accumulates as a byproduct. Each cycle leaves a dated record of what was scanned and what was found. Assessors ask for exactly that.
Exposure becomes rankable. When every store carries a sensitivity profile, the public bucket holding customer records sorts above the private one holding build logs.

The trade-off is worth stating plainly. Automation buys cadence and coverage, and it does not buy precision on any single object, because the scan read a sample. Treat a profile as a reliable statement about a resource and a weak one about any file inside it. Broader cloud data security practice covers what happens once the profile exists.

Industries and Use Cases for Sensitive Data Discovery

The obligation to know where data lives is written into the rules governing regulated industries. That is why discovery budgets survive the cost reviews that kill other security projects. HIPAA safeguards, payment card rules, and privacy law each define a different scope of data to find. The health data framework shows how far that scope reaches in one sector.

Industry	Regulated data type	Obligation that drives discovery	Discovery-specific difficulty
Healthcare	Protected health information	HIPAA Security Rule safeguards over all ePHI a covered entity creates, receives, maintains, or transmits	Research and analytics copies of clinical data spread across project accounts
Payments and retail	Cardholder and account data	PCI DSS Requirement 12.5.2, which requires identifying all locations where account data is stored	Account data reaching logs, backups, and file exports outside the defined environment
Financial services	Customer records and transaction data	GDPR Article 30 records of processing, plus sector reporting rules	Long retention periods keep decades of records in systems that changed owners
SaaS and technology	Customer data held on behalf of tenants	Contractual commitments and regional data residency terms	Tenant data copied into shared analytics stores during product development
Public sector	Citizen records and case files	Statutory retention and disclosure duties	Legacy systems that predate the cloud accounts now hosting their exports

Where Regulation Sets the Scope

Two obligations do most of the work. GDPR Article 30 says each controller “shall maintain a record of processing activities under its responsibility.” That record must describe the categories of data subjects and the categories of personal data. A record naming categories of personal data is a data inventory under another name, and the GDPR cloud framework covers the rest of the obligation.

PCI DSS v4.0.1, published in June 2024, is more specific. Requirement 12.5.2 requires scope to be documented and confirmed at least once every 12 months, including “Identifying all locations where account data is stored, processed, and transmitted.” It names locations outside the defined environment, applications that process cardholder data, transmissions between systems, and file backups. Service providers confirm every six months.

Residency adds a second axis. A multinational estate replicates the same customer record into several regions. A team then has to scope the sensitive data discovery control Asia-Pacific, European, and North American operations each need separately. Working out which regime applies to which copy is the first job in multi-cloud compliance, and it starts from an inventory that records where each copy sits.

Enforcing Policies and Managing Data Exposure

A finding is worth exactly as much as the control it triggers. A discovery program stalls at this step when the finding stops at a report. The objection is familiar: last year’s scan produced a spreadsheet nobody used. Two causes produce it: the scan was a point-in-time snapshot of a changing estate, and its findings carried no exposure context, so nothing in them could be ranked or assigned.

From Finding to Enforced Control

Turning a profile into an enforced policy runs through five steps, and skipping one returns you to the spreadsheet.

Attach exposure to the finding. A store of customer records reachable from the internet is a different problem from the same store behind a private endpoint. Identity reach counts as exposure too, so a role half the organization can assume belongs in the same calculation.
Resolve an owner. An account tag, a team, or a service catalogue entry will do. A finding with no owner has no next step, and it will still be open at the next audit.
Choose the control the finding earns. Remove public access, tighten the role, encrypt with a managed key, apply a retention rule, move the data, or delete the copy. Stopping exfiltration once access is settled is the job of data loss prevention.
Write the rule, not the ticket. One bucket fixed is a ticket. A policy that blocks public access on any store profiled as holding regulated data is a control, and it covers the next store before anyone finds it.
Verify on the following cycle. The scan that proves the control held is the only evidence the loop closed.

What to Measure

Coverage is the honest headline metric. The number that matters is the share of your estate a discovery tool reaches, broken down by account, region, and service type. Amazon publishes a documentation page for assessing automated discovery coverage, because the answer is never all of it. A tool reporting 3,000 findings across 40% of your accounts is in worse shape than one reporting 300 across 95%.

Three more numbers are worth tracking. Time from resource creation to first profile shows whether cadence matches the rate your teams create storage. Share of findings with a resolved owner predicts how many will ever be actioned. Re-exposure rate, meaning stores that were remediated and drifted back, tells you whether you fixed a bucket or wrote a rule.

Feed all four into the wider data risk management program instead of reporting them as a standalone scan result.

How Orca Finds and Contextualizes Sensitive Cloud Data

Start with what Orca is not. Orca is not a data catalogue, so it does not maintain the business glossary or the stewardship workflow an analytics team runs. It is not a network DLP enforcement point inspecting traffic that leaves your environment either. Orca sits where this article ends, at the point a discovered store becomes a ranked, owned, enforceable risk.

Orca’s Unified Data Model continuously maps every asset, configuration, identity, network path, and data store into a single model. It then prioritizes risk across severity, asset exposure, blast radius, and data sensitivity. A sensitive store arrives with the identities that can reach it and the paths an attacker could take. It lands ranked instead of sitting at row 412 of a report.

Collection runs agentless-first through SideScanning™, which reads workloads’ runtime block storage without requiring an agent on each one, with the Orca Sensor supplying runtime signal. That is the out of band model described earlier, with its documented limits, applied across accounts rather than workload by workload. Get a demo to see which of your data stores carry exposure nobody has ranked yet.

Frequently Asked Questions About Sensitive Data Discovery

How Often Should a Discovery Scan Run

Treat the regulatory cadence as a floor, not a design. PCI DSS asks for scope confirmation every 12 months, and every six for service providers. Neither number says anything about how fast your teams create storage, so match the interval to that rate instead. Environments where developers provision managed databases weekly need daily profiling of new resources, while a static estate under change control can run less often and stay current.

Why Do Two Tools Report Different Sensitive Data Counts for the Same Estate

Because they enumerated different resources and sampled different amounts of each. One product may cover object storage and managed databases while another adds warehouses, file shares, and snapshots. Sampling depth widens the gap, since a scanner reading 300 rows and one reading 5,000 will disagree about a column holding regulated values in a minority of rows. Compare the resource inventories the two tools built before comparing their finding counts.

Can Discovery Find Sensitive Data in Backups and Snapshots

Sometimes, and it is the coverage question worth asking early. Backup vaults and snapshots are separate resource types, and a tool that profiles a live database will not automatically profile its snapshots. Ask a vendor directly which backup services its product enumerates in each cloud. The orphaned backup whose retention outlived the policy governing its source is easy to overlook and hard to reach.

What Does Sensitive Data Discovery Cost to Run

Amazon Macie publishes three billing dimensions: the number of S3 buckets evaluated, the number of objects monitored for automated discovery, and the quantity of data inspected. Targeted scans add standard S3 GET and LIST request charges on top of that. Cost therefore tracks how many resources enter scope as much as how deeply each one gets read. A broad profiling pass and a deep targeted scan price differently, so scope both before committing.

Who Should Own Sensitive Data Discovery Internally

There is no settled answer, and the split depends on the organization. Security teams usually own the tooling and the findings, data platform teams own the stores and the fixes, and privacy or compliance owns the record a regulator reads. What consistently fails is leaving ownership unnamed, since a finding with no owner sits open regardless of which team should have taken it.

Sensitive Data Discovery: Tools, Best Practices & How It Works

Key Takeaways

What is Sensitive Data Discovery?

Discovery, Inventory, and Classification Are Three Different Jobs