CoreQ Academy1.1 Compare and contrast various AI types and techniques used in cybersecurity.
To secure AI you need to know what kind of AI you are dealing with and how it was trained. A spam classifier, a fraud model and a chat assistant fail in very different ways.
Types of AI
- Generative AI
- Models that create new content (text, images, code, audio) from patterns learned in training data. Useful for drafting and summarising; can also produce convincing false output.
- Machine learning
- Systems that learn patterns from data instead of following hand-written rules, e.g. a model that learns what malicious URLs look like.
- Statistical learning
- The mathematical foundation of ML: fitting models (regression, classification, clustering) to data and measuring uncertainty. Many security analytics such as anomaly scores are statistical.
- Transformers
- A neural-network architecture that uses attention to weigh how parts of the input relate to each other. It powers most modern language models.
- Deep learning
- Machine learning with many-layered neural networks. Learns complex features from raw data such as images, audio or packet sequences; needs lots of data and compute.
- Natural language processing (NLP)
- Techniques for computers to understand and produce human language: classifying phishing emails, extracting indicators from reports, translation.
- Large language models (LLMs)
- Very large transformer models trained on huge text corpora. Capable general assistants, but expensive, prone to hallucination and exposed to prompt injection.
- Small language models (SLMs)
- Compact language models that can run on a laptop, phone or private server. Cheaper and easier to keep data in-house; less capable on broad tasks.
- Generative adversarial networks (GANs)
- Two networks trained against each other: a generator makes fakes and a discriminator tries to spot them. Used for synthetic data and, maliciously, deepfakes.
Model training techniques
- Model validation
- Checking a trained model on data it has not seen to measure real-world accuracy and catch overfitting.
- Supervised learning
- Training on labelled examples (malicious vs. benign). Common for classifiers such as malware detection.
- Unsupervised learning
- Finding structure in unlabelled data, such as clustering similar alerts or spotting outliers for anomaly detection.
- Reinforcement learning
- An agent learns by trial and error, receiving rewards or penalties. Also used to tune LLM behaviour from human feedback (RLHF).
- Fine-tuning
- Further training a pre-trained model on a smaller, task-specific dataset. Cheaper than training from scratch; the fine-tuning data becomes a security asset and a poisoning target.
- Epoch
- One complete pass through the training data. Too many epochs can overfit; too few underfit.
- Pruning
- Removing weights or neurons that contribute little, making the model smaller and faster with limited accuracy loss.
- Quantization
- Storing model weights with fewer bits (for example 8-bit instead of 32-bit) to cut memory and speed up inference, sometimes reducing accuracy.
Prompt engineering
- System prompts
- Instructions set by the application that define the model behaviour and boundaries. Must not contain secrets: assume users can extract them.
- User prompts
- The input a user sends. Treat as untrusted data.
- One-shot prompting
- Giving the model one worked example of the task before the real input.
- Multi-shot prompting
- Giving several examples (few-shot) so the model follows a pattern or output format more reliably.
- Zero-shot prompting
- Asking the model to do a task with no examples, relying on its general training.
- System roles
- Message roles (system, user, assistant) that tell the model which instructions carry authority. Role separation is a first defence against injected instructions.
- Templates
- Fixed prompt structures with slots for user data. They keep instructions consistent and make it harder for user text to rewrite the task.
Check your understanding
Loading quiz…
Original CoreQ Academy material, mapped to objective 1.1 of the publicly available CompTIA SecAI+ CY0-001 exam objectives (Exam objectives V1, document version 1.1). Not affiliated with or endorsed by CompTIA. Contains no exam questions.