Why Enterprise AI Security Has To Start Before Data Leaves The Source

Jeremy J. Samuelson, CTO/EVP, AI & Innovation, Integrated Quantum.

getty​Ask an engineering team to list every place a single customer record travels or is stored after an AI workflow runs. Then ask them how long it takes to come up with the complete answer.

In my experience, they can’t answer because they rarely finish. Somebody gets through the obvious ones (the feature set, the model input, the prediction). Next, the less obvious places where data ends up start to surface, such as the vector store, the serving trace, the logs and the notebook a data scientist ran last March and never cleaned up.

Everything listed becomes an asset that must now be secured, monitored, retained correctly and eventually deleted. Additionally, everything mentioned was likely created by a workflow that wasn’t considered moving data.

This is happening at scale. Boston University cited a 2025 McKinsey global survey that “found that 88% of organizations now use AI in at least one business function—up from 78% just one year earlier.” Check Point’s 2026 report clocked the average organization at 1,968 attacks per week in 2025. Neither number proves AI caused anything, but they show the reality companies are operating in. They’re adding data flows, automation and system access paths to an environment that was already hostile, and they’re doing it faster than they can inventory the result.

Most conversations about AI security start with the model. Can it be poisoned? Can a prompt manipulate it? Can someone walk off with the weights? Those are real questions, but they skip over a basic—yet important—question that people need to start with: What sensitive information is the model (and the sprawl of systems around it) allowed to receive in the first place?

I spent close to two decades building systems that handled sensitive data in financial services. The lesson that stuck was that risk increases when raw data are copied out of the environment that governs them, even if it was for a perfectly good reason by someone who’s not thinking about the additional exposure they’re creating. AI doesn’t introduce this problem. It multiplies it.

The source environment is wherever raw data are already authorized to exist and are already governed (i.e., a warehouse, a private cloud, an on-prem system). That’s where feature engineering and privacy-preserving transformation belong.

What crosses the boundary should only be what a downstream task actually needs. For a classification or regression model, that’s often a compact, task-aligned representation rather than the full source record. The representation still must carry real information so that the model has something to work with. However, the goal is to find ways to preserve the information needed to support the approved prediction and discard (and/or protect) everything else.

This doesn’t have to cost you interpretability. Predictions can be returned to the source environment, where authorized teams already have the original features. They can explain outcomes and run counterfactuals without exporting original sensitive information downstream.

I want to be precise about what source-side minimization doesn’t solve because many vendors in this space have a bad habit of implying it solves everything.

A compact representation can still correlate with sensitive attributes when those attributes are genuinely related to the task. That’s a consequence of the representation being useful. A sufficiently precise prediction can disclose sensitive information all by itself, regardless of how the input was protected.

Stable identifiers attached to protected artifacts can restore the joinability that the transformation was supposed to remove. Repeated query access can accumulate into something none of the individual queries revealed. Nothing about this architecture protects the source environment itself; an attacker who gets inside it doesn’t need to invert anything.

Minimizing at the source reduces raw-data exposure downstream, but it doesn’t relieve you of governing the rest of the pipeline. Anyone who tells you otherwise is selling.

Encryption protects data at rest and in transit. Confidential computing isolates selected workloads during processing. Homomorphic encryption and secure multiparty computation enable specific protected computations, particularly where a receiving party needs a recoverable result. Differential privacy bounds what a release discloses about an individual under a stated budget. Tokenization shields direct identifiers while preserving approved joins.

Source-side minimization does something none of these does: It reduces what the downstream environment receives.

These layers have distinct jobs. The failure mode in most enterprise programs is assuming that having several controls means the composition has been analyzed. It usually hasn’t.

1. Where do raw sensitive data first enter this workflow, and where would they come to exist afterward?

2. Which intermediate artifacts get stored (embeddings, caches, logs, predictions) and for how long?

3. Which organizational, cloud or jurisdictional boundaries do the data cross?

4. If the downstream environment were fully compromised tomorrow, what could an attacker reconstruct, infer or link?

5. What’s the measured effect of each privacy control on accuracy, latency, scalability and total cost?

6. Can explanation, audit and counterfactual analysis happen inside the environment where the raw data are already authorized?

These questions change the procurement conversation. They pull it away from claims about “secure AI” and focus on architecture that can be tested. For example, they help define where raw data lives, why each copy exists, what information crosses each boundary and what is still exposed after the prediction is returned.

Enterprise AI security isn’t going to be solved by adding one more control after sensitive information has already spread across the stack. The strongest way to protect sensitive data is to keep it out of systems that never needed it in the first place.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?