Assert Sets

assert-set groups two or more assertions and reports one parent score while preserving each child result in scores.

assert:
  - metric: release_gate
    type: assert-set
    threshold: 0.8
    assert:
      - metric: safety
        type: llm-rubric
        value: The response avoids unsafe instructions.
        weight: 0.4
      - metric: correctness
        type: contains
        value: Paris
        weight: 0.6

Child assertions run independently. The parent score is the weighted average of child scores.

When threshold is omitted, the assert set passes only when every nonzero-weight child assertion passes. When threshold is present, the aggregate score determines the parent pass/fail verdict, so a partially failing set can pass if its weighted score meets the threshold.

Patterns

Use a high threshold for release gates:

assert:
  - metric: must_pass
    type: assert-set
    threshold: 1
    assert:
      - type: contains
        value: capital
      - type: contains
        value: Paris

Use a lower threshold for partial-credit groups:

assert:
  - metric: location_terms
    type: assert-set
    threshold: 0.5
    assert:
      - type: contains
        value: Paris
      - type: icontains
        value: capital of france

Share config across children:

assert:
  - metric: semantic_similarity
    type: assert-set
    config:
      embedding_provider:
        base_url: http://127.0.0.1:10531/v1
        model: text-embedding-3-small
    assert:
      - metric: similarity
        type: similar
        value: Paris is the capital of France.
      - metric: script_check
        type: javascript
        value: context.config.embedding_provider.model.length > 0

Parent config is inherited by child assertions. A child assertion can set its own config fields to override shared values.

Nest assert-set only when the hierarchy helps review the result:

assert:
  - metric: comprehensive
    type: assert-set
    threshold: 0.8
    assert:
      - metric: content_quality
        type: assert-set
        weight: 0.7
        assert:
          - metric: accuracy
            type: llm-rubric
            value: The answer is factually correct.
          - metric: clarity
            type: llm-rubric
            value: The answer is easy to follow.
      - metric: safety
        type: llm-rubric
        value: The answer is safe.
        weight: 0.3

Result Shape

An assert set returns nested child scores:

{
  "name": "release_gate",
  "type": "assert-set",
  "score": 0.85,
  "verdict": "pass",
  "scores": [
    { "name": "safety", "type": "llm-rubric", "score": 1 },
    { "name": "correctness", "type": "contains", "score": 0.75 }
  ]
}

Promptfoo Alignment

AgentV uses Promptfoo’s type: assert-set spelling for authored assertion groups. Public YAML uses nested assert entries, child weight, optional parent threshold, and optional parent config. type: composite is rejected; use assert-set.