Today, many organizations struggle to adequately test generative AI applications to ensure they can move smoothly from prototype to production. One of the main obstacles is the difficulty of creating a robust test database that covers a wide range of potential risks, including emerging threats like jailbreak attacks. Even when quality data is available, the assessment process can be complex and labor-intensive, often making it difficult for development teams to decipher the results and develop effective countermeasures.
To address these obstacles, we are pleased to announce the public preview of automated security assessments in Azure AI Studio. These assessments are designed to assess an application’s vulnerability to jailbreak attempts and its propensity to generate content that includes violent, sexual, self-harming, and hateful themes. Additionally, each assessment comes with natural language explanations, helping developers understand the metrics and make informed decisions about mitigations.
Developers have the flexibility to evaluate their applications using their own test datasets or use pre-built competitor alert templates developed by Microsoft Research to generate high-quality test data. Additionally, this feature enables Azure AI Studio to improve and accelerate manual red teaming efforts by enabling red teams to create and automate competitor alerts at scale.
Prerequisites
To evaluate with AI-powered metrics, you need:
A test dataset in .jsonl format. See the next section for dataset requirements
An implementation of one of these models: GPT 3.5 models, GPT 4 models, or Davinci models.
Supported scenarios and datasets
We currently offer support for these scenarios:
Question Answering: This scenario is intended for applications that involve answering user queries and providing answers.
Conversational: This scenario is suitable for applications where the model converses using an extended search approach to extract information from the documents you provide and generate detailed answers.
Source:
https://learn.microsoft.com/en-us/
https://azure.microsoft.com/en-us/updates