Write golden tests

A golden test is a request and the answer you expect, written in the ruleset itself. The check runs every test, and so does every engine in its conformance run. A test is how you say what the rule means, so a reviewer reads the tests before the rule.

Hands-on

1. Write one test per outcome you care about: the allowed case, the denied case, and the client channel when the rule runs in a form. These are the tests of orders.ruleset.yaml:

   tests:
     - name: ten items are allowed and announced
       entity: Order
       operation: create
       given:
         data: { id: o-1, quantity: 10 }
       expect:
         decision: allow
         findings: []
         commands: [order.placed]
   
     - name: eleven items are denied and nothing is announced
       entity: Order
       operation: create
       given:
         data: { id: o-2, quantity: 11 }
       expect:
         decision: deny
         findings:
           - { rule: order.quantity.max, fields: [/quantity], message: You can order at most 10 items. }
         commands: []
   
     - name: the browser checks the quantity too
       entity: Order
       operation: create
       channel: client
       given:
         data: { id: o-3, quantity: 11 }
         trigger: change
       expect:
         decision: deny
         findings:
           - { rule: order.quantity.max }

2. Prove that a test can fail. Change the second test to expect allow for eleven items, and run the check:

   $ rcas check orders.ruleset.yaml
   shop.orders@1.0.0  sha256:4583c90bbfca...  2 rules (1 client-safe), 1 params
     FAIL  eleven items are allowed
           decision "deny" != "allow"
           findings [order.quantity.max] != []
     3 golden tests, 1 failed
   [exit status 1]

   The check names the test, says what differed, and exits with status 1. Put the test back.

3. Try a test in the playground. The playground runs the golden tests when you press Check.

   ▶ Try it YourselfRuns in your browser with the TypeScript engine. Nothing to install.

Done when

• Each rule has at least one test where it raises a finding and one where it does not.
• A test that expects commands lists them (commands: [order.placed]), and a denied one expects commands: [].
• A deliberately wrong expectation makes the check fail.

Go deeper

• Golden tests: every field of a test and how the answer is matched.
• Ship a rule change, step "Add golden tests".