93.3% precision, 6.7% false-positive rate
Counted from the recorded dispositions of the sample below and from nothing else — the verdicts two named officers wrote, each quoted over the same denominator, stated beside it. Your own marks do not move these two. They are laid over the record and recounted on the line under the tiles, labelled as your session’s and kept in your own browser — because this demo has no database, and a figure a visitor can change is not the institution’s measurement.Your session’s recount. The figures above are the record: they are counted from dispositions two named officers wrote, and nothing you do on this page moves them. Mark a row and a second pair appears here — the record’s verdicts with your marks laid over them, recomputed as you go. Your marks are kept in this browser and stamped with the time you made them, so they are still here when you come back; they reach no server and no other reader, because this demo has no database.
Precision. Of the detections in the adjudicated sample, the share a named officer marked a true positive. It describes the flags this control raised, over the sample stated in the sampling rule — not the control’s behaviour on traffic it never flagged.
False-positive rate of flagged-and-reviewed. The complement of precision over the same sample and the same denominator: the share of reviewed flags marked wrong. It is not a false-positive rate over the negative population, which would need a count of unflagged submissions.
Recall — sensitive submissions detection missed. No figure on any surface measures it, and none can be derived from the adjudicated sample: adjudication only ever sees events the engine already flagged. It would take: Ground truth independent of the detector — a second, differently-built classifier over the same traffic, or a human review of a sample of unflagged interactions.
Coverage — the share of staff and devices the sensor reaches. The pilot has no deployment inventory in this record, so a detection count has no population behind it. It would take: An endpoint inventory reconciled against the staff directory, per week.
The adjudication queue is a stratified sample of the detections in the trailing thirty days: each data class contributes in proportion to its share of the window, and within a class events are taken in a fixed order derived from the pilot seed. Detections separately reported as errors are included as well.
Reported errors are in the sample by construction rather than by the draw, so precision over this sample is a floor on the sample’s own accuracy and not an unbiased estimate of the control’s. Recall is not measured at all — see the accuracy record.
Who marked itDraws alternate between the two adjudicators in the sampled order, so neither officer marks a contiguous block and the sample carries both signatures.
M. Halloran: Owns the control and proposes the policy versions. Marks the odd-numbered draws. Y. Abadi: Independent reviewer: does not propose policy and does not own the console. Marks the even-numbered draws.
16 detections nobody has looked at yet
The next draw, by the same rule as the 120 below — which is the point of drawing it the same way: a precision figure this quarter is only comparable with next quarter’s if the two samples were taken alike. These carry no verdict, and the figures above are computed without them, because an unreviewed detection is not evidence of anything.| Excerpt | Classes | Decision | Adjudication | |
|---|---|---|---|---|
| What does a returned item on account 00408779 do to the overdraft fee schedule?B. Mwangi, Finance · Jul 17, 2026 at 9:34 AM | Account Numbers | Monitorv3 | read-only — this principal does not adjudicate | Chain |
| Reconcile these two entries against deposit account 00863164 and tell me which one is the duplicate posting.L. Tran, Retail & Branch Ops · Jul 23, 2026 at 4:55 PM | Account Numbers | Monitorv4 | read-only — this principal does not adjudicate | Chain |
| Customer Espen Mackintosh is disputing three ACH debits on account 00849552. Draft a response that explains the provisional credit timeline.N. Oyelaran, Credit Administration · Jul 27, 2026 at 10:16 AM | Member PII, Account Numbers | Warnv4 | read-only — this principal does not adjudicate | Chain |
| Draft the recommendation section of a credit memorandum for Andre Asante, requested facility $518,612, secured by equipment.N. Oyelaran, Credit Administration · Jul 29, 2026 at 3:09 PM | Credit Memos, Member PII | Warnv4 | read-only — this principal does not adjudicate | Chain |
| Rewrite the mitigants paragraph in the attached credit memorandum for Alicia Qureshi at Zanesville Enterprises, requested $1,313,374. Keep the committee language, drop the hedging.N. Oyelaran, Credit Administration · Jul 29, 2026 at 3:52 PM | Credit Memos, Member PII | Warnv4 | read-only — this principal does not adjudicate | Chain |
| This call keeps returning 401. The header is Authorization: Bearer ghp_aym2myuy2yuiqeuuuqaum2eyyiua — what am I missing?G. Iyer, IT · Jul 30, 2026 at 8:19 AM | API Keys / Creds | Blockv4 | read-only — this principal does not adjudicate | Chain |
| What does covenant 6(b) in loan CL-2026-50577 actually require at each quarter end?T. Beaulieu, Commercial Lending · Aug 3, 2026 at 8:41 AM | Loan Files | Monitorv5 | read-only — this principal does not adjudicate | Chain |
| Write a follow-up note to Camille Nilsen at (419) 555-0149 about the certificate of deposit maturing next month.A. Bergström, Credit Administration · Aug 4, 2026 at 4:15 PM | Member PII | Warnv5 | read-only — this principal does not adjudicate | Chain |
| What is the collateral position on loan CL-2026-06494? The borrower is Marisol Krishnamurthy, and the appraisal came in at $2,180,680.R. Okafor, Commercial Lending · Aug 4, 2026 at 5:54 PM | Loan Files, Member PII | Redactv5 | read-only — this principal does not adjudicate | Chain |
| Summarize this complaint. Customer: Greta Nakashima, callback (419) 555-0103, branch: Alder Creek Main.H. Delacroix, Credit Administration · Aug 6, 2026 at 4:20 PM | Member PII | Warnv5 | read-only — this principal does not adjudicate | Chain |
| How should I phrase a beneficiary change confirmation for Nikhil Ramanathan, reachable on (419) 555-0182?W. Nyathi, Compliance/BSA · Aug 7, 2026 at 1:17 PM | Member PII | Monitorv5 | read-only — this principal does not adjudicate | Chain |
| Reconcile these two entries against deposit account 00117740 and tell me which one is the duplicate posting.P. Nakamura, Retail & Branch Ops · Aug 7, 2026 at 4:43 PM | Account Numbers | Monitorv5 | read-only — this principal does not adjudicate | Chain |
| Please review this loan application for Wren Callahan, SSN 453-56-4914, account #00196680. The debt-to-income ratio is 49% — is that inside our commercial guideline?D. Whitfield, Commercial Lending · Aug 11, 2026 at 8:25 AM | Member PII, SSN / Tax ID, Account Numbers | Redactv5 | read-only — this principal does not adjudicate | Chain |
| Summarize the credit file for loan CL-2026-68449 — borrower Bela Nilsen, guarantor on the Kettering-Freight Systems relationship. Focus on covenant history and past-due notes.A. Bergström, Credit Administration · Aug 11, 2026 at 2:38 PM | Loan Files, Member PII | Warnv5 | read-only — this principal does not adjudicate | Chain |
| What is the collateral position on loan CL-2026-52110? The borrower is Marisol Krishnamurthy, and the appraisal came in at $807,448.A. Bergström, Credit Administration · Aug 14, 2026 at 9:42 AM | Loan Files, Member PII | Warnv5 | read-only — this principal does not adjudicate | Chain |
| The card ending in this number keeps declining: 4539860236701693. The customer is Imani Kovalenko. What should the branch check first?A. Bergström, Credit Administration · Aug 14, 2026 at 4:26 PM | Account Numbers, Member PII | Warnv5 | read-only — this principal does not adjudicate | Chain |
A codename in the institution’s dictionary, firing on a landscaping quote
“Project Sycamore” is an M&A codename at Alder Creek Community Bank and a tree everywhere else. The bank chose the word, the dictionary detector found it exactly where the bank said to look, and the detection is still wrong — which is what a custom dictionary costs and why it is configuration rather than code.Draft a polite reply to the landscaping vendor about the Project Sycamore planting schedule — the Main Street beds are behind and we want them finished before the branch anniversary.
proved by a checksum or a format rule · matched by a pattern, which is a shape, or by the taxonomy or the institution’s dictionary, which are vocabulary markers — neither is a proof
It is one of the 8 errors in the 120-event sample, and it is in the denominator of the 6.7% above rather than excused out of it. Open its chain
112 events marked true positive
The worked half of the same draw: the two figures at the top are counted from these verdicts and from nothing else. Read the excerpt, read the spans, mark it. Your marks are kept in your own browser, stamped with the time you made them, so the work you did here is still here next time — this demo has no database, so they reach no server and no other reader, and the queue says which of the two it is rather than showing a control that quietly saves nothing.| Excerpt | Classes | Decision | Recorded verdict | Adjudication | |
|---|---|---|---|---|---|
| Summarize the borrower file below for underwriting. The applicant is Gunnar Odenkirk, SSN 246-37-1102, deposit account 00739815, requested amount $698,428.R. Okafor, Commercial Lending · Jul 31, 2026 at 9:56 AM | Member PII, SSN / Tax ID, Account Numbers | Warnv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Extract every reporting deadline from the agreement for loan CL-2026-72490 and put them in a table.T. Beaulieu, Commercial Lending · Jul 31, 2026 at 9:12 AM | Loan Files | Monitorv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Summarize this complaint. Customer: Devrim Okonjo, callback (419) 555-0107, branch: Alder Creek Main.L. Tran, Retail & Branch Ops · Jul 31, 2026 at 8:48 AM | Member PII | Monitorv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Draft a birthday message for Hassan Achterberg, date of birth 1965-03-04, who has banked with us for eleven years.H. Delacroix, Credit Administration · Jul 30, 2026 at 3:43 PM | Member PII | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Explain the settlement timing for this card: 4539743403600627. The merchant says it authorized on Friday.J. Kowalczyk, Retail & Branch Ops · Jul 30, 2026 at 1:25 PM | Account Numbers | Monitorv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| How should I phrase a beneficiary change confirmation for Caleb Beauchamp, reachable on (419) 555-0158?N. Oyelaran, Credit Administration · Jul 30, 2026 at 11:30 AM | Member PII | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Confirm the wire instructions for Alicia Farrimond — routing 040178329, account 00647569. Draft the callback script.T. Beaulieu, Commercial Lending · Jul 29, 2026 at 5:29 PM | Member PII, Account Numbers | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Reconcile these two entries against deposit account 00907808 and tell me which one is the duplicate posting.B. Mwangi, Finance · Jul 29, 2026 at 2:22 PM | Account Numbers | Monitorv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Reconcile these two entries against deposit account 00248892 and tell me which one is the duplicate posting.L. Tran, Retail & Branch Ops · Jul 28, 2026 at 4:55 PM | Account Numbers | Monitorv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| What is the collateral position on loan CL-2026-33798? The borrower is Tomas Asante, and the appraisal came in at $573,568.D. Whitfield, Commercial Lending · Jul 28, 2026 at 1:12 PM | Loan Files, Member PII | Warnv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| How should I phrase a beneficiary change confirmation for Clemence Villareal, reachable on (419) 555-0146?J. Kowalczyk, Retail & Branch Ops · Jul 28, 2026 at 11:38 AM | Member PII | Monitorv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Write a follow-up note to Fiona Prieto at (419) 555-0173 about the certificate of deposit maturing next month.P. Nakamura, Retail & Branch Ops · Jul 28, 2026 at 10:26 AM | Member PII | Monitorv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Summarize the credit file for loan CL-2026-66961 — borrower Wren Winterbourne, guarantor on the Kettering-Freight Systems relationship. Focus on covenant history and past-due notes.M. Ramaswamy, Commercial Lending · Jul 27, 2026 at 4:41 PM | Loan Files, Member PII | Warnv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Verify this routing number before I release the wire: 046002693. It came in on a faxed instruction.K. Osei, Commercial Lending · Jul 27, 2026 at 12:34 PM | Account Numbers | Monitorv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| This call keeps returning 401. The header is Authorization: Bearer sk-live-aiaaei2uyum6i2iy2q22ai2uqi2q — what am I missing?G. Iyer, IT · Jul 27, 2026 at 10:36 AM | API Keys / Creds | Blockv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Tighten the risk-rating rationale in this credit memorandum so it reads plainly for the loan committee. Borrower is Astrid Callahan, Portage-Ridge Partners.N. Oyelaran, Credit Administration · Jul 24, 2026 at 12:59 PM | Credit Memos, Member PII | Warnv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Is it safe to keep this in the deployment config? sk-live-ee2ee6u6aaea62ueiyqie2iu6i2q It is the sandbox key for the core integration.V. Sokolov, IT · Jul 24, 2026 at 11:14 AM | API Keys / Creds | Blockv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Draft the recommendation section of a credit memorandum for Priya Odenkirk, requested facility $1,101,256, secured by equipment.H. Delacroix, Credit Administration · Jul 24, 2026 at 11:13 AM | Credit Memos, Member PII | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Draft a birthday message for Emiliano Fontenot, date of birth 1991-10-25, who has banked with us for eleven years.T. Beaulieu, Commercial Lending · Jul 24, 2026 at 11:11 AM | Member PII | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| The card ending in this number keeps declining: 4539711258703895. The customer is Devrim Vasquez. What should the branch check first?I. Quintero, Retail & Branch Ops · Jul 24, 2026 at 8:28 AM | Account Numbers, Member PII | Monitorv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Summarize the credit file for loan CL-2026-06521 — borrower Astrid Espinosa, guarantor on the Kettering-Freight Systems relationship. Focus on covenant history and past-due notes.K. Osei, Commercial Lending · Jul 24, 2026 at 8:09 AM | Loan Files, Member PII | Warnv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| What is the collateral position on loan CL-2026-47946? The borrower is Tobias Hollingsworth, and the appraisal came in at $564,892.H. Delacroix, Credit Administration · Jul 23, 2026 at 3:57 PM | Loan Files, Member PII | Warnv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Turn the covenant table in the Hocking-Hills Manufacturing credit memorandum into plain sentences a relationship manager can read out. Fixed-charge coverage is 1.57x.T. Beaulieu, Commercial Lending · Jul 23, 2026 at 2:11 PM | Credit Memos | Monitorv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Draft a birthday message for Priya Krishnamurthy, date of birth 1991-04-17, who has banked with us for eleven years.A. Bergström, Credit Administration · Jul 23, 2026 at 1:04 PM | Member PII | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Tighten the risk-rating rationale in this credit memorandum so it reads plainly for the loan committee. Borrower is Yusuf Nilsen, Hocking-Hills Manufacturing.T. Beaulieu, Commercial Lending · Jul 23, 2026 at 12:51 PM | Credit Memos, Member PII | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| Write a hold notice for account 00014111, held by Emiliano Fontenot. Deposit is a business check over the case-by-case threshold.B. Mwangi, Finance · Jul 23, 2026 at 11:47 AM | Account Numbers, Member PII | Monitorv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| What does a returned item on account 00737427 do to the overdraft fee schedule?C. Ferrante, Finance · Jul 22, 2026 at 5:10 PM | Account Numbers | Monitorv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |
| I am preparing an annual review on loan CL-2026-71292. Borrower Caleb Obradovic, operating company Wapakoneta-Auto Group. What questions should the committee expect?N. Oyelaran, Credit Administration · Jul 22, 2026 at 3:54 PM | Loan Files, Member PII | Warnv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Write a hold notice for account 00205627, held by Tobias Benali. Deposit is a business check over the case-by-case threshold.E. Lindqvist, Finance · Jul 22, 2026 at 1:28 PM | Account Numbers, Member PII | Monitorv4 | True positiveM. Halloran | read-only — this principal does not adjudicate | Chain |
| Confirm the wire instructions for Jamal Villareal — routing 042990961, account 00114945. Draft the callback script.K. Osei, Commercial Lending · Jul 22, 2026 at 9:25 AM | Member PII, Account Numbers | Warnv4 | True positiveY. Abadi | read-only — this principal does not adjudicate | Chain |