Microsoft has released PyRIT, a generative AI tool designed for red teams.

On February 23, 2024, Microsoft released an open source automation tool called PyRIT for preemptive risk identification in generative artificial intelligence (AI) systems. PyRIT is a risk identification tool designed for generative AI systems. It is an open source framework to help test for unfavorable conditions in an AI system. Microsoft notes that the development of PyRIT is a demonstration of its commitment to democratize AI security for its customers, partners and colleagues.

Unlike traditional adversity testing, adversity testing of generative AI systems should include identification of both safety risks and risks of responsible use of the AI, such as incorrectness or unreliability of the resulting content. PyRIT's design provides abstraction and extensibility for future development of its capabilities. The tool implements five interfaces: target, dataset, evaluation engine, attack strategies, and memory.

PyRIT supports integration with models from Microsoft Azure OpenAI service, Hugging Face, and Azure Machine Learning Managed Online Endpoint. The tool supports two styles of attack strategy: single-rotation and multi-rotation.

The first strategy involves sending a combination of "breaking" and malicious suggestions to an AI system and evaluates its response. In the second strategy, the system sends a combination of "breaking" and malicious suggestions to the AI system and then responds to the AI system's response based on the resulting score. The first approach is faster, while the second approach is a more realistic adversary and allows for more sophisticated attack strategies.

"PyRIT is not just a tool for generating prompts; it changes tactics based on the response of the generative AI system and generates the next input for the generative AI system. This automation continues until the task outlined for the security professional is achieved"," reads the announcement released by Microsoft. Microsoft noted that the tool does not replace manual adversity testing of generative AI systems."PyRIT was created in response to our belief that sharing resources for AI adversity testing across the industry is beneficial to all. We encourage our colleagues in the industry to familiarize themselves with this tool and consider using it for adversity testing of their own generative AI applications"," the announcement reads.