What NIST's generative AI guidance says about building trustworthy systems

A fluent answer is not automatically a trustworthy one. Organizations using generative artificial intelligence must consider how a system behaves when it produces unsupported information, uses sensitive data or operates where a mistake could have serious consequences. The National Institute of Standards and Technology offers voluntary guidance for evaluating such risks.

NIST published its AI Risk Management Framework in January 2023 and a generative AI companion profile in July 2024. The latter, identified as NIST AI 600-1, addresses issues arising when a system generates text, images, code or other content. The document is a tool for managing risks, not an automatic legal certification or binding requirement for every developer.

Trustworthiness is broader than whether a chatbot gets a test question right. Reliability, safety, security, privacy, transparency and fairness all matter, as does the purpose for which a system is used. A tool that performs acceptably as an optional brainstorming aid might be unsuitable for a setting where people cannot easily challenge its outputs.

That is why evaluation must continue after a product launches. Teams need to identify potential harm, examine who could be affected, test realistic use cases, document limitations and monitor what happens as circumstances change. A single demonstration can show a feature working; it cannot establish how the system will perform in every situation.

Generative systems create a particular problem for source verification. An output can look authoritative while offering a mistaken statement or a nonexistent citation. Users should be able to separate generated claims from independently checked evidence. A newsroom, for example, cannot turn a polished draft into a verified news report merely by publishing it.

Human oversight matters, but its design matters too. A reviewer needs the information, time and authority to spot a problem and stop or correct it. Organizations also need useful routes to report failures and update systems when new risks emerge, rather than treating their safeguards as permanent.

The lesson is not that generative AI is inherently useless or automatically safe. NIST's approach encourages evidence-based decisions about benefits and harms, together with meaningful accountability over time. The choice of deployment context can be as important as the model itself.

Reporting note: This article summarizes NIST's published risk-management framework and does not evaluate or certify a particular AI product.