National Cybersecurity
Association of Ukraine

Responsible use of AI in cyber defence.

Guide G9 · Published

For security teams, IT staff and managers who are considering artificial intelligence (AI) tools for defence or already use them. Where they genuinely help, where they fail, and which rules to put in place before first use.

Who this is for and when to use it

It is written for people responsible for security: administrators, analysts, and managers of small organisations and NGOs. Use it when you are deciding whether to connect an AI tool to your defences (alerts, logs, mail), or when you already use one and want to put rules around it.

It is about using AI for defence. For protecting AI systems themselves, see “AI security checklist for small organisations” and “Agentic AI: least privilege and human approval”.

Where AI genuinely helps

AI works best as an assistant that narrows the workload for a person, not as the one who decides. CISA itself gives a useful example. It uses unsupervised machine learning to find anomalies in critical-infrastructure network data and in the logs of federal civilian networks. The system only highlights potential anomalies; analysts review them and decide which alerts to investigate first 3.

  • Confidence scoring of threat indicators: a model assigns a score for the reliability and completeness of submitted indicators. It helps analysts prioritise review but does not replace it 3.
  • Spotting personal data in threat-sharing submissions: a model flags suspect passages and an analyst confirms or rejects the finding. Regular audits help keep the process trustworthy 3.

What these examples share is that AI shows where to look and a person makes the decision. The UK NCSC advises starting from your own needs, understanding how the technology works, and only then deciding whether a tool will give a real gain in security 4.

For summarising logs and analysing suspicious emails we give no effectiveness claims: this guide has no reliable source for them. If you try AI on such tasks, treat the output as a lead and check it against the original.

Errors and over-trust

Any detection tool is wrong in two directions. A false positive is an alarm over a harmless event, and it costs analysts time. A false negative is a real attack that goes unflagged, and it stays unnoticed.

Generative models add another risk. NIST calls it confabulation: confidently stated but false content 2. The NCSC also notes that large language models can produce false information 5. OWASP describes over-reliance: users do not verify an answer, and the error slips into important decisions 9. NIST calls this automation bias, an excessive deference to automated systems, and notes that it amplifies other risks 2.

A model’s summary or conclusion is a lead, not evidence. Before you act, open the source: the log itself, the email itself, the file itself.

The AI tool as a target, and data leaks

If a model reads an email, a web page or a log, that content can carry hidden instructions. OWASP calls this indirect prompt injection and notes that no fully reliable way to prevent it is known 6. NIST writes that researchers have already shown such attacks stealing data or remotely running malicious code 2. For a defender, the uncomfortable consequence is that the data you analyse may have been written by the attacker.

If you let a tool act (block an account, isolate a device, delete emails), a mistake or a planted instruction becomes an action. OWASP links excessive agency to hallucinations and prompt injection, and advises requiring human approval for high-impact actions and granting minimal permissions 8.

Data is a separate matter. OWASP counts personal data, financial and health records, credentials and confidential documents as sensitive information, and advises teaching users not to submit it to models 7. The NCSC notes that AI systems should work without exposing sensitive data to unauthorised parties 5. Joint guidance from CISA, NSA, the FBI and partners stresses the role of data security in the accuracy, integrity and trustworthiness of AI outcomes 10. Security logs and mail can easily contain exactly such data.

Until you know where the data goes, do not paste logs, emails, keys, passwords or personal data into an external AI tool.

Practical rules

  1. 01Start with one narrow need. State what exactly should get better, and only then choose a tool 4.
  2. 02A person decides every action with consequences: blocking, deleting, isolating, notifying outside parties. The AI proposes and a person confirms 8. The NIST AI RMF calls for policies that define the roles of people and AI and the oversight of the system 1.
  3. 03Name an owner for each tool. The NCSC advises leaders to know who is accountable for AI security and to have a plan for when a system’s integrity, availability or confidentiality is compromised 5.
  4. 04Measure errors on your own data. Before launch, compare the tool’s output with your specialists’ decisions: how many false positives and how many missed events. The NIST AI RMF calls for performance to be assessed under conditions similar to real use and checked regularly in operation 1. Decide in advance the threshold at which the tool is switched off.
  5. 05Data going in. Do not pass sensitive data to external models without rules and an approved supplier. Anonymise or mask it before use 7.
  6. 06Separate and label untrusted content (emails, pages, logs). Do not let it trigger actions on its own 6.
  7. 07Check against the source, especially when the stakes are high 9.
  8. 08Keep a way to switch it off. The NIST AI RMF calls for mechanisms to supersede, disengage or deactivate an AI system that behaves inconsistently with its intended use 1. Keep a log of what the tool does.
  9. 09For tools that act on their own, also read the joint CISA and partners guidance on the careful adoption of agentic AI 11.

Questions to put to a vendor

This is NAK’s editorial list, built from the sources above. It is not a standard and not a guarantee of security.

  • Exactly which of my data leaves my environment, where is it kept and for how long, how is it deleted, and can I opt out of its use for training 7?
  • How did you measure errors (false positives and missed events), and on what data? Can the measurement be repeated on my data 1?
  • What happens when the input contains someone else’s instructions? How does the tool separate them from mine 6?
  • Which actions can the tool take by itself, with what permissions, and where in that chain is human approval required 8?
  • How do I switch it off, and what is kept in the logs 1?
  • Who is accountable, and who fixes the system if it fails 4?

Limits

This is a guide, not legal advice, not a certification and not a service. NAK does not assess AI tools, check your systems, or offer detection or analysis services. The CISA examples show its own practice; they do not prove that a similar tool will suit you. The NCSC guidance on intelligent security tools is dated 2019, and the NCSC marks it as under review 4. This text was last checked on 24 Oct 2026; the sources used are listed below.

Sources

  1. 1NIST AI Resource Center. NIST AI RMF Core: Govern, Map, Measure, Manage · accessed 2026-10-24
  2. 2NIST. Artificial Intelligence Risk Management Framework: Generative AI Profile (NIST AI 600-1) · accessed 2026-10-24
  3. 3CISA. CISA Artificial Intelligence Use Cases · accessed 2026-10-24
  4. 4UK National Cyber Security Centre. Intelligent security tools: assessing intelligent tools for cyber security · accessed 2026-10-24
  5. 5UK National Cyber Security Centre. AI and cyber security: what you need to know · accessed 2026-10-24
  6. 6OWASP GenAI Security Project. LLM01:2025 Prompt Injection · accessed 2026-10-24
  7. 7OWASP GenAI Security Project. LLM02:2025 Sensitive Information Disclosure · accessed 2026-10-24
  8. 8OWASP GenAI Security Project. LLM06:2025 Excessive Agency · accessed 2026-10-24
  9. 9OWASP GenAI Security Project. LLM09:2025 Misinformation · accessed 2026-10-24
  10. 10CISA, NSA, FBI and partners. AI Data Security: best practices for securing data used to train and operate AI systems (AA25-142A) · accessed 2026-10-24
  11. 11CISA and partners. Careful Adoption of Agentic Artificial Intelligence (AI) Services · accessed 2026-10-24