Skip to main content

Annotation filter

Has the annotations produced upstream reviewed by a decision model, which estimates for each one the probability that it is relevant given the passage it cites. Annotations below the minimum score are dropped; the others carry on unchanged. Useful to cut false positives from a verbose agent without rewriting it.

Not to be confused with annotation validation, which corrects citations but never removes anything.

Ports​

  • Annotations (left, annotation type) — one or more outputs of agents, supervisors or other annotation nodes. The filter waits for all its inputs before running.
  • Document (top, content type, optional) — otherwise, the text the annotations were produced on is used.
  • Output (right, annotation type) — the annotations kept.

Configuration​

  • Decision model (required).
  • Minimum score — minimum probability to keep an annotation. 50% by default.
  • Criterion (optional) — the question asked for each annotation. Empty: "Is this annotation relevant and justified given the cited passage?". A more precise criterion ("does it point at a real legal risk?") filters more finely.

Behavior​

  • Each annotation is judged with an excerpt of the document around the cited passage, which keeps calls short and cheap.
  • Nothing is lost on failure: if the model doesn't answer for an annotation, it is kept. With no model configured, every annotation goes through.
  • The node details (test panel) list the dropped annotations with their probability, to tune the threshold.
  • Consumption is charged to Finances (decision models).