Clik here to view.
Improving Bash Generation in Small Language Models with Grammar-Constrained...
Bash is one of the most flexible and powerful interfaces exposed to AI agents. In the right system, a model that emits , , , or a shell pipeline is producing an executable action that can read files,...
View ArticleClik here to view.
Updating Classifier Evasion for Vision Language Models
Advances in AI architectures have unlocked multimodal functionality, enabling transformer models to process multiple forms of data in the same context. For instance, vision language models (VLMs) can...
View ArticleClik here to view.
Practical LLM Security Advice from the NVIDIA AI Red Team
Over the last several years, the NVIDIA AI Red Team (AIRT) has evaluated numerous and diverse AI-enabled systems for potential vulnerabilities and security weaknesses before they reach production. AIRT...
View ArticleClik here to view.
Why CVEs Belong in Frameworks and Apps, Not AI Models
The Common Vulnerabilities and Exposures (CVE) system is the global standard for cataloging security flaws in software. Maintained by MITRE and backed by CISA, the program gives each vulnerability a...
View ArticleClik here to view.
Structuring Applications to Secure the KV Cache
When interacting with transformer-based models like large language models (LLMs) and vision-language models (VLMs), the structure of the input shapes the model’s output. But prompts are often more than...
View ArticleClik here to view.
Sandboxing Agentic AI Workflows with WebAssembly
Agentic AI workflows often involve the execution of large language model (LLM)-generated code to perform tasks like creating data visualizations. However, this code should be sanitized and executed in...
View ArticleClik here to view.
Defending AI Model Files from Unauthorized Access with Canaries
As AI models grow in capability and cost of creation, and hold more sensitive or proprietary data, securing them at rest is increasingly important. Organizations are designing policies and tools, often...
View ArticleClik here to view.
Secure LLM Tokenizers to Maintain Application Integrity
This post is part of the NVIDIA AI Red Team’s continuing vulnerability and technique research. Use the concepts presented to responsibly assess and increase the security of your AI development and...
View ArticleClik here to view.
AI Red Team: Machine Learning Security Training
At Black Hat USA 2023, NVIDIA hosted a two-day training session that provided security professionals with a realistic environment and methodology to explore the unique risks presented by machine...
View ArticleClik here to view.
Analyzing the Security of Machine Learning Research Code
The NVIDIA AI Red Team is focused on scaling secure development practices across the data, science, and AI ecosystems. We participate in open-source security initiatives, release tools, present at...
View ArticleClik here to view.
Mitigating Stored Prompt Injection Attacks Against LLM Applications
Prompt injection attacks are a hot topic in the new world of large language model (LLM) application security. These attacks are unique due to how malicious text is stored in the system. An LLM is...
View ArticleClik here to view.
NVIDIA AI Red Team: An Introduction
Machine learning has the promise to improve our world, and in many ways it already has. However, research and lived experiences continue to show this technology has risks. Capabilities that used to be...
View ArticleClik here to view.
Evaluating the Security of Jupyter Environments
How can you tell if your Jupyter instance is secure? The NVIDIA AI Red Team has developed a JupyterLab extension to automatically assess the security of Jupyter environments. jupysec is a tool that...
View ArticleClik here to view.
Improving Machine Learning Security Skills at a DEF CON Competition
Machine learning (ML) security is a new discipline focused on the security of machine learning systems and the data they are built upon. It exists at the intersection of the information security and...
View Article
/pic9814888.png)
/fit-in/580x348/filters:format(webp)/english-betterindia/media/media_files/2026/09/17/balaghat-3-2026-09-17-18-46-40.png)