Secure AI and LLM Deployment

What is a secure AI and LLM deployment

A secure AI and LLM deployment means introducing an AI-based system into the organisation — an assistant with access to documents, a RAG (Retrieval-Augmented Generation) search engine, an agent that calls tools or a model that supports decisions — in a way that covers risk assessment, data access control, security testing and model oversight from the first stage.

A language model accepts instructions in natural language and cannot fully distinguish instructions from data. If it has access to documents, email or company systems, each of those integrations is a new channel through which an attacker can influence its behaviour or extract information. That is why security and compliance are not a stage bolted on after deployment — they are part of the project, from system classification to monitoring after go-live.

Secure deployment is one of four services in the AI systems protection pillar on the AI Security page, alongside the AI/LLM application pentest, the shadow AI assessment and AI Act readiness.

Most common risks when deploying LLMs

We describe the risks of applications built on language models using the OWASP Top 10 for LLM Applications, 2025 edition. In a deployment we most often design safeguards against:

• LLM01:2025 Prompt Injection — instructions injected by the user or hidden in content the model reads: documents, emails, web pages.
• LLM02:2025 Sensitive Information Disclosure — disclosure of personal data, trade secrets or documents the requester has no access to.
• LLM04:2025 Data and Model Poisoning — poisoning of the data the model is fine-tuned on, or of the knowledge base it uses.
• LLM05:2025 Improper Output Handling — passing model output unchecked to a browser, a shell or another system.
• LLM06:2025 Excessive Agency — giving the model tools and permissions broader than the task requires.
• LLM07:2025 System Prompt Leakage — leakage of the system prompt together with business rules and secrets.
• LLM08:2025 Vector and Embedding Weaknesses — weaknesses of the vector database in RAG systems, including missing separation of permissions between users.

The remaining categories — LLM03:2025 Supply Chain, LLM09:2025 Misinformation and LLM10:2025 Unbounded Consumption — are addressed when selecting the model and provider, in answer verification procedures and in usage limits.

Governance and AI Act compliance

The AI Act — Regulation (EU) 2024/1689 — imposes obligations that depend on the system’s risk category and the organisation’s role. That is why a deployment starts with classification: is the system prohibited, high-risk, subject to transparency obligations or free of specific requirements — and is the organisation its provider or its deployer.

On that basis we prepare documentation, human oversight rules and a risk management process. We structure the AI management system according to ISO/IEC 42001, which describes roles, policies and the life cycle of AI systems in an organisation.

The timeline for high-risk requirements was changed by the Digital Omnibus — Regulation (EU) 2026/1744. Requirements for systems listed in Annex III apply from 2 December 2027, and for systems covered by Annex I from 2 August 2028 (as of 7 October 2026). The obligation to support AI literacy (Art. 4) already applies.

A full compliance assessment — system inventory, gap analysis and action plan — is carried out as part of the AI Act Readiness and AI Governance service. AI governance is part of the broader area of governance, risk and compliance.

Data protection and access control

In practical LLM deployments most of the risk comes from what data the model can access and on whose behalf it acts. In the deployment architecture we apply:

• Data minimisation — only the data needed for the task reaches the model; personal and sensitive data is classified before it enters the knowledge base.
• User-aligned permissions — the RAG system returns only documents the requester is entitled to in the source systems.
• Limited tools — an agent gets only the tools and permissions necessary for the task, and operations with effects in other systems require human confirmation.
• Environment isolation — separation of test and production environments and of data belonging to different teams or clients.
• Logging and audit — logs of prompts, responses and tool calls that make it possible to reconstruct system behaviour after an incident.
• Model provider terms — verification of where data is processed, how long it is retained and whether it may be used to train the model.

If employees already use public AI tools without the organisation’s approval, the starting point is a shadow AI assessment and an AI use policy.

Technologies

We match the platform to the organisation’s needs so that it meets requirements on data processing location and access control. The choice depends on the client’s infrastructure and use case; example options:

• IBM watsonx — a platform with components for building models (watsonx.ai), data (watsonx.data) and model oversight (watsonx.governance); deployable in the cloud or on your own infrastructure.
• Azure OpenAI — OpenAI models delivered as part of Microsoft Azure services, integrated with the identity and network of the Azure environment.
• Amazon Bedrock — access to foundation models from various providers within the AWS environment.

The platform is a technology option, not the goal of the project. The same controls — classification, permissions, testing and monitoring — apply regardless of the model provider chosen.

Who it is for

A secure AI and LLM deployment is for organisations that:

• plan an assistant or search engine based on internal documents, email or customer data,
• build AI agents that perform operations in company systems,
• operate in regulated sectors and must demonstrate control over data processed by the model,
• want to establish whether a planned system will be high-risk within the meaning of the AI Act,
• have completed an AI pilot and need a security assessment before going into production.

How we work

1. Risk assessment and classification. We inventory use cases, data sources and user roles, and establish the AI Act risk category and the organisation’s role.
2. Architecture and access control. We design the solution with data minimisation, role-based permissions, environment isolation and limited model tools.
3. Pre-production security testing. We carry out an AI/LLM application pentest against the OWASP Top 10 for LLM Applications, followed by a retest after fixes.
4. Governance and monitoring. We prepare policies, documentation and human oversight rules, and after go-live we monitor model behaviour and the security impact of changes.

Related terms

• AI Act — the EU regulation on artificial intelligence
• AI security — protecting artificial intelligence systems
• Shadow AI — use of AI tools without the organisation’s approval
• Access controls — rules for granting permissions to resources
• IAM — identity and access management
• Artificial intelligence — the basics of the concept
• Knowledge base: AI and cybersecurity — articles on threats and defence