AI Security Glossary

Adversarial Machine Learning

The study and practice of attacks against machine learning systems — including techniques to fool, manipulate, or extract information from AI models.

AI Copilot Security

Security considerations for AI-powered copilot tools — coding assistants, productivity AI, and enterprise AI assistants — that are integrated into sensitive business workflows.

AI Governance

The policies, processes, and accountability structures an organisation uses to ensure AI is developed and used responsibly, safely, and in compliance with applicable laws.

AI Hallucination

When an AI model generates confident, plausible-sounding output that is factually incorrect, fabricated, or not grounded in reality.

AI Red Teaming

The practice of systematically testing AI systems by attempting to make them behave harmfully, unsafely, or in ways that circumvent their intended guidelines.

AI Risk Management

The process of identifying, assessing, and mitigating risks associated with developing or deploying AI systems — from technical failures to legal exposure and ethical harms.

Data Poisoning

An attack that corrupts an AI model's training data to manipulate its behaviour — causing misclassifications, backdoors, or degraded performance.

Deepfakes

AI-generated synthetic media — video, audio, or images — that realistically portray a person doing or saying something they never did, used increasingly in fraud and social engineering.

Generative AI

A category of AI that can create new content — text, images, code, audio, and video — by learning patterns from training data.

Large Language Model (LLM)

A type of AI model trained on vast amounts of text data that can generate, summarise, translate, and reason about natural language — powering tools like ChatGPT, Claude, and Gemini.

Model Exfiltration

An attack that steals a proprietary AI model — either its weights directly or a functional replica — by querying the model repeatedly and learning from its responses.

Prompt Injection

An attack against AI systems that manipulates the model's behaviour by embedding malicious instructions within input data, overriding intended behaviour.

Retrieval-Augmented Generation (RAG)

An AI architecture that combines a language model with a retrieval system, grounding responses in specific documents rather than relying solely on training knowledge.

Shadow AI

The unauthorised or undisclosed use of AI tools by employees — often outside IT oversight — creating data security, compliance, and operational risks.