Terminology

Name	Definition
Attack	A Message sent to a Target intended to cause malfunction.
Attack Category	The category of an Attack Objective. For example, Violence, Bodily Harm, and Terrorism and General Hate Speech & Harassment.
Attack Method	How the attack tries to accomplish the objective. Prompt injection, data poisoning, role assignment, obfuscation etc.
Attack Objective	The goal of a Conversation(s). For example, system prompt exfiltration, executing unauthorized actions, executing unauthorized tasks, etc.
Code of Conduct (CC)	Set of rules and guidelines that the target system should adhere to, determined by the user.
Conversation	Multiple Interactions between attacker and target system.
Interaction	A single Message and a Target Response.
Judge	An LLM system that determines the success or failure of a given Attack
Message	A single message to an LLM that may or may not have contextual history of a conversation and may or may not be considered an Attack.
Red team	A group of people authorized and organized to emulate a potential adversary’s attack or exploitation capabilities against an enterprise’s security posture.
Red teaming	The practice of testing a system with an adversarial approach to assess that system's defenses
Session	A red teaming exercise against a Target. Consists of a collection of Conversations corresponding to an Attack Objective.
Single-shot Attack	A one-time message sent to a Target intended to cause malfunction.
Target	An LLM or LLM application/system.
Target Response	A response to a Message from a Target.
Vulnerability	A successful attack on a target that led to a target response deemed as unintended, undesired, or a malfunction.

Session Types

Fortify has 4 Session Types: Essential, Target-Specific, Comprehensive, and Custom.

Essential Sessions

Essential Sessions consist of our general list of 45 out of 153 Attack Objectives that apply across domains and use cases. Each Essential Session is configured to serve 5 Conversations per Attack Objective, and 5 Messages per Conversation.

That equates to Total Attacks Messages = 45 Objectives x 5 Attacks x (Select Attack Replay Count) for a single Essential Session.

Target-Specific Sessions

Target-Specific Sessions leverage a Target's Code of Conduct (requires user input) to extract custom Attack Objectives that apply to the Target's use-case or domain (Target-Exclusive Attack Objectives).

We will also identify any relevant Standard Attack Objectives from our list of 153 that apply to the Target's use-case or domain. So, a Target-Specific Session is configured to serve 5 Conversations per Attack Objective, and 5 Messages per Conversation.

The number of Attack Objectives, and therefore the total number of Attack Messages, in a Target-Specific Session will depend on the number of Target-Exclusive and Standard Attack Objectives extracted from the Code of Conduct.

Comprehensive Sessions

Comprehensive Sessions provide the most comprehensive testing coverage by combining Essential and Target-Specific Sessions. Specifically, a Comprehensive Session includes all 153 Standard Attack Objectives as well as all of the extracted Target-Exclusive Attack Objectives.

The total number of Attack Objectives, and therefore the total number of Messages, in a Comprehensive Session will depend on the number of Target-Exclusive Attack Objectives extracted from the Target's Code of Conduct.

Total Attacks Messages = 153 Objectives x 5 Attacks x (Select Attack Replay Count)

Custom Sessions

Custom Sessions provide the most flexibility in configuration, enabling users to select specific Attack Categories or individual Attack Objectives and configuring the number of Conversations and Messages from 1 through 10 for each.

Session Ontology

All Attack Objectives share the same default structure (though this can be modified with the Custom Session Type):

Attack Objective {N}

Each Attack Objective has 5 Conversations, each containing 5 Interactions:

Conversation {C} (where C ranges from 1 to 5)

Interaction {I} (where I ranges from 1 to 5)

Message
Response

Example

Attack Objective 1

Conversation 1
- Interaction 1
  - Message
  - Response
- Interaction 2
  - Message
  - Response
- ...
- Interaction 5
  - Message
  - Response
Conversation 2
- Interaction 1
  - Message
  - Response
- ...
- Interaction 5
  - Message
  - Response

... and so on until Conversation 5.

Attack Objective 2

(Same structure as above)

...

Attack Objective 153 (or more with Target-Specific or Comprehensive Session Type)

(Same structure as above)