CoreQ Academy My progress Sign in
SecAI+ level · Domain 2.0 Securing AI Systems · 40% of the mapped exam

2.2 Given a set of requirements, implement security controls for AI systems.

12 topics · 4 quiz questions

AI controls sit in two places: on the model itself (evaluation and guardrails) and in front of it (a gateway that inspects, limits and authorises requests).

Model controls

Model evaluation
Testing the model against safety, accuracy and adversarial test sets before and after changes.
Model guardrails
Rules and classifiers around the model that block disallowed topics, sensitive data or unsafe actions in inputs and outputs.
Prompt templates
Fixed prompt structures that separate trusted instructions from user data, reducing injection and keeping behaviour consistent.

Gateway controls

Prompt firewalls
A filtering layer that inspects prompts and responses for injection attempts, secrets, personal data and policy violations.
Rate limits
Caps on requests per user or key over time; slow down abuse, scraping and model extraction.
Token limits
Maximum input and output tokens per request; controls cost and limits denial-of-wallet attacks.
Input quotas
Allowances on how much a user can submit over a period.
Data size
Limits on the size of uploads or context submitted.
Quantity
Limits on the number of items, files or calls.
Modality limits
Restricting input types (text only, no images or audio) to shrink the attack surface, since hidden instructions can ride in images or files.
Endpoint access controls
Authentication and authorisation on model APIs so only approved apps and users can call them.

Guardrail testing and validation

Guardrail testing and validation
Regularly attacking your own guardrails with jailbreak and injection test suites (red teaming) and confirming they still block what they should after every model or prompt change.

Check your understanding

Loading quiz…

Original CoreQ Academy material, mapped to objective 2.2 of the publicly available CompTIA SecAI+ CY0-001 exam objectives (Exam objectives V1, document version 1.1). Not affiliated with or endorsed by CompTIA. Contains no exam questions.