Monitor

Two signals tell you how the rules behave in production. The checksum says which rules made a decision; log it with every evaluation so an audit can replay it. The finding code RULE-EVALUATION-ERROR says a rule could not be evaluated: the engine failed closed and denied the operation. Count it and alert on it, because it means a user was denied for a reason that is not a business rule.

Hands-on

1. Log the checksum at start-up and after every reload. The rule server already does:

   $ RULE_SERVER_TOKEN=learn-demo-token RULES_DIR=./rules node packages/server/dist/main.js
   server log: {"level":"info","message":"ruleset loaded","id":"shop.orders.eu","version":"1.0.0","checksum":"sha256:b31b76fa5a5b8755a2f6df8de0e3d4152c78fc9a2d2c86df6703d51abda1d5c2","source":"document"}
   server log: {"level":"info","message":"ruleset loaded","id":"shop.orders","version":"1.0.0","checksum":"sha256:4583c90bbfcacb54bf78e11d9e8190ecc086ce4b69aade609b7a0928d452ea82","source":"document"}

2. Send a request the rule cannot evaluate: the quantity arrives as the string "11", and lte compares only numbers.

   > POST /evaluations
   > Authorization: Bearer learn-demo-token
   < 200
   {
     "ruleset": "shop.orders",
     "version": "1.0.0",
     "checksum": "sha256:4583c90bbfcacb54bf78e11d9e8190ecc086ce4b69aade609b7a0928d452ea82",
     "decision": "deny",
     "findings": [
       {
         "rule": "order.quantity.max",
         "code": "RULE-EVALUATION-ERROR",
         "severity": "error",
         "message": "This rule could not be evaluated.",
         "fields": [],
         "blocking": true,
         "status": "open",
         "resolution": "none",
         "source": "shop.orders@1.0.0",
         "detail": "number expected, got \"11\""
       }
     ],
     "effects": [],
     "commands": []
   }

   The decision is deny, the finding code is RULE-EVALUATION-ERROR, and detail says why. The answer carries the checksum, so you know which rules were running.

3. In your service, write one log line per evaluation with the ruleset id, version, checksum and decision, and the codes of the findings. Alert when the share of RULE-EVALUATION-ERROR findings rises above zero for a ruleset.

Done when

• Every evaluation can be traced to a checksum.
• An alert fires on RULE-EVALUATION-ERROR, and it names the rule (rule) and the reason (detail).
• After a roll-out, the logs show the new checksum on every instance.

Go deeper

• Incident: RULE-EVALUATION-ERROR: triage, roll back, fix forward.
• Fail closed: when the engine raises it.