Key takeaways

  • Shadow AI is AI use outside your approved governance and oversight.
  • Retention, training, and access vary by provider and product configuration.
  • Evaluate risks and responsibilities before allowing sensitive information into a tool.

Shadow AI is the use of AI tools outside an organization’s approved oversight and controls. It can create risks when employees use personal accounts, upload sensitive files, connect business systems, or rely on unverified output without a reviewed process.

For defense contractors, the key question is whether the tool and its use are appropriate for the information and requirements involved. Not every AI product trains on customer inputs, and not every AI use is automatically a CMMC violation. Provider terms, configuration, system scope, and actual handling matter.

The following ten risks are structured review prompts, not claims that every tool has the same behavior.

1. Sensitive information sent to an unapproved service

Risk: prompts or attachments can disclose CUI, FCI, personal data, source code, or proprietary designs beyond an approved environment.

Safeguard: define allowed information types and tools. Prohibit sensitive uploads until security, contractual, and provider requirements have been reviewed.

2. Personal accounts outside access governance

Risk: the organization may lack visibility into account ownership, authentication, sharing, and removal when someone leaves.

Safeguard: use approved account management and access controls. Document who can use the service and how access is removed.

3. Unclear retention and training terms

Risk: prompts, files, and responses may be retained or used in ways that differ across plans and settings.

Safeguard: review the relevant terms and administrative settings. Record retention, deletion, training use, and provider access rather than assuming all products behave alike.

4. Connectors with excessive permissions

Risk: an AI assistant or integration can access more email, files, or systems than the task requires.

Safeguard: review connector permissions and scopes. Use least privilege, approve integrations, and remove unnecessary access.

5. Prompt injection through untrusted content

Risk: malicious instructions in documents or external content can influence a system that processes them, particularly when it can use tools or access data.

Safeguard: treat retrieved content as untrusted, constrain tool actions, and require appropriate review for consequential operations.

6. Inaccurate output treated as authoritative

Risk: a fluent answer may contain fabricated facts, incorrect citations, or unsuitable compliance advice.

Safeguard: verify important statements against authoritative sources. Require qualified review before relying on output for security configurations, assessments, or contractual decisions.

7. Unsafe code or automation

Risk: generated scripts can introduce vulnerabilities, expose secrets, or make damaging changes if executed without review.

Safeguard: apply your normal code review, testing, and change approval process. Limit privileges and test in an appropriate environment first.

8. Missing logs and evidence

Risk: unmanaged use can leave the organization unable to explain who accessed information, what was shared, or how an incident occurred.

Safeguard: establish logging and evidence requirements appropriate to the use case. Confirm the provider can support your operational and assessment needs.

9. Unclear provider responsibilities

Risk: subcontractors, providers, and internal teams may assume someone else owns data protection, incident handling, or review.

Safeguard: document shared responsibilities and evaluate the service against applicable contract and cloud requirements before use.

10. No approved path for useful AI work

Risk: employees may work around a blanket prohibition when they do not understand the rules or have a safe alternative.

Safeguard: provide approved use cases, role-based training, and a process for requesting new tools. Review the inventory as products and responsibilities change.

A starting point for AI governance

Create an AI-use inventory, classify the information involved, identify accountable owners, and establish approval and review rules. Connect AI governance to your existing access management, incident response, training, and supplier review processes.

For a focused learning program, see tailored cybersecurity training. Training is one part of governance, alongside technical controls and operating procedures.

Sources and further reading

Back to articles