Generative AI Security for Enterprises: Prompt Injection, Data Leakage & Controls (2026)

Part of our Japan AI governance cluster. Japan’s AI law is soft, but the security risks of the AI you deploy under it are not.

Most generative-AI security conversations I sit in start in the wrong place — model accuracy, hallucinations, “is the output any good.” Those are quality problems. The security problems are different, more concrete, and increasingly well-catalogued. The OWASP Top 10 for LLM Applications, updated for 2025, is the closest thing the industry has to a shared map, and it is where I start any enterprise GenAI review.

I work in information security at a Japanese enterprise (CISSP, CCSP). This is the short version of that review, with the Japanese regulatory overlay that foreign teams tend to miss.

The three risks that matter most first

The 2025 OWASP LLM Top 10 reworked the list to reflect real-world incidents and the rise of agentic AI (OWASP). Three risks deserve your attention before the rest:

1. Prompt injection (the #1 risk). Untrusted input — a user message, a web page your model reads, a document in a RAG pipeline — manipulates the model into ignoring its instructions. It holds the top position because it is the most fundamental LLM vulnerability and potentially the hardest to fully prevent. You manage it; you do not “solve” it.

2. Sensitive information disclosure. The model surfaces confidential data — from its prompt, its context, its training, or a connected source — to someone who should not see it. This is the risk that turns an AI feature into a data-breach vector.

3. System-prompt leakage (new in 2025). Added after real incidents, this is the extraction of your system prompt — which often contains internal rules, filtering criteria, permission structures, and decision logic. Treat anything in a system prompt as potentially exposable, not as a secret store.

To these, the 2025 list adds RAG-specific weaknesses: vulnerabilities in how vectors and embeddings are generated, stored, or retrieved can allow content injection, output manipulation, or unauthorized data access.

Why these are hard — the security-design reading

The uncomfortable truth is that an LLM does not cleanly separate instructions from data. Everything is tokens. That single design fact is why prompt injection resists a clean fix: there is no architectural boundary between “the system’s rules” and “the attacker’s input” the way there is between code and a parameterized query.

So the principles I apply are the same ones I apply everywhere, adjusted for that reality:

• Least privilege for the model’s reach. An LLM is only as dangerous as the data and actions it can touch. Scope its access; do not wire a chat assistant to every system “to be helpful.”
• Assume the boundary will fail. Since instruction/data separation is porous, design for the injection that will land — constrain what a compromised model can do, rather than trusting it not to be compromised.
• Measure the model’s behavior. You cannot manage what you cannot see. Log prompts, outputs, and tool calls; monitor for the signatures of injection and data egress.

Enterprise controls that contain the risk

Not a silver bullet — a defense-in-depth stack (OWASP):

• Constrain privilege and actions. Give the model the minimum data access and the minimum tool/action set its function requires. Put high-impact actions behind human approval.
• Sanitize training and fine-tuning data. Scrub or mask sensitive content before it enters a model you control, so it cannot be surfaced later.
• Filter input and output. Define sensitive categories; apply semantic filters and string checks to scan for disallowed content in both directions.
• Harden RAG pipelines. Validate and access-control the content that enters your vector store; untrusted documents are an injection vector.
• Keep secrets out of system prompts. Assume the prompt can leak; do not put credentials, raw permission logic, or secrets there.
• Monitor in real time. Watch for prompt-injection attempts, sensitive-data egress, and output manipulation as live signals someone owns.

The Japanese regulatory overlay

Here is the part foreign teams under-weight. Japan’s AI Promotion Act is soft law with no penalties — but the security failures above do not live in the AI law. They live in the binding law underneath it:

• If your generative-AI feature leaks personal data of people in Japan, the APPI breach-notification duties can trigger — the preliminary report in days, the final report within 30 (or 60). The model being “the cause” changes nothing.
• The AI Guidelines for Business set a comply-or-explain expectation that you have governed exactly these risks. “We deployed an LLM with no data controls” is not an explanation that survives contact with a Japanese customer’s security review.

So the security work and the compliance work are the same work: contain prompt injection and data disclosure, and you simultaneously satisfy the soft AI guidance and stay clear of the hard APPI line.

References

• OWASP Top 10 for LLM Applications 2025 (OWASP, confirmed 2026-06-11)
• OWASP Top 10 for LLMs — project page (OWASP, confirmed 2026-06-11)
• Japan AI Safety Institute (J-AISI official, confirmed 2026-06-11)