Golden tests

Put the expected decisions next to the rules, so every engine proves it gives the same answer.

A golden test is a request and the answer it must get. It lives in the ruleset, under tests, so the rules and their proof travel together. rcas check runs them, and so does every runtime's conformance suite: an engine that disagrees fails.

given holds the request: data, and optionally original, actor, ctx, resolutions, locale, trigger and view. expect holds the decision and the complete list of findings. Each expected finding names its rule and may add code, severity, blocking, status, fields, message and location; only the members you write are compared.

Syntax

one golden test

tests:
  - name: eleven items are denied      # what a reviewer reads
    entity: Order
    operation: create
    given:
      data: { quantity: 11 }
    expect:
      decision: deny
      findings:
        - { rule: order.quantity.max, fields: [/quantity], message: You can order at most 10 items. }

Example

Three tests cover a denial, an allowed order with an info finding, and an update. Every one of them passes, or this page would not build.

golden-tests.ruleset.yaml

ruleCascade: 1.0.0
kind: RuleSet
metadata: { id: learn.golden-tests, version: 1.0.0, title: Golden tests }
scope:
  - { level: organization, id: learn }
entities:
  Order:
    schema: { $ref: "./learn.openapi.yaml#/components/schemas/Order" }
rules:
  - id: order.quantity.max
    kind: validation
    target: { entity: Order, field: /quantity }
    operations: [create, update]
    assert: { op: lte, args: [{ var: data.quantity }, 10] }
    severity: error
    finding: { code: LRN-TST-001, message: order.tooMany }
  - id: order.quantity.min
    kind: validation
    target: { entity: Order, field: /quantity }
    operations: [create, update]
    assert: { op: gte, args: [{ var: data.quantity }, 1] }
    severity: error
    finding: { code: LRN-TST-002, message: order.tooFew }
  - id: order.notes.missing
    kind: validation
    target: { entity: Order, field: /notes }
    operations: [create]
    assert: { op: exists, args: [{ var: data.notes }] }
    severity: info
    finding: { code: LRN-TST-003, message: order.noNotes }
messages:
  en:
    order.tooMany: "You can order at most 10 items."
    order.tooFew: "Order at least 1 item."
    order.noNotes: "No delivery notes."
tests:
  - name: eleven items are denied
    entity: Order
    operation: create
    given:
      data: { quantity: 11, notes: "Leave at the door" }
    expect:
      decision: deny
      findings:
        - rule: order.quantity.max
          code: LRN-TST-001
          severity: error
          blocking: true
          fields: [/quantity]
          message: You can order at most 10 items.
  - name: three items without notes are allowed, with an info finding
    entity: Order
    operation: create
    given:
      data: { quantity: 3 }
    expect:
      decision: allow
      findings:
        - { rule: order.notes.missing }
  - name: an update is checked too
    entity: Order
    operation: update
    given:
      data: { quantity: 20 }
      original: { quantity: 2 }
    expect:
      decision: deny
      findings:
        - { rule: order.quantity.max }

request.json

{
  "entity": "Order",
  "operation": "create",
  "data": {
    "quantity": 11,
    "notes": "Leave at the door"
  }
}

Result, from the engine

Decisiondeny1 finding, server channel

LRN-TST-001errorblockingYou can order at most 10 items./quantity

Findings
Code	Severity	Fields	Message	Blocking
`LRN-TST-001`	error	`/quantity`	You can order at most 10 items.	yes

Try it YourselfOpens this ruleset and request in the playground. Nothing to install.

In the playground, press Check to run all the golden tests. Change 11 to 9 in the first test and press Check again: the test fails and says why.

Common mistakes

Leaving out a finding. findings is the complete set. A test that expects one finding fails when the engine reports two, even if the extra one is only info.
Testing only the happy path. Write at least one test that allows and one that denies for every rule.
Vague names. The name is what a reviewer reads in a failing build. Say what the request is and what must happen.

Exercise

Add a golden test named "an empty order is denied". It sends 0 items (with notes) and expects one finding from order.quantity.min with the code LRN-TST-002.

Start in the playground

Hint

Copy the first test. Change the name, the quantity and the expected rule.

Show answer