93.3% precision, 6.7% false-positive rate
Counted from the recorded dispositions of the sample below and from nothing else — the verdicts two named officers wrote, each quoted over the same denominator, stated beside it. Your own marks do not move these two. They are laid over the record and recounted on the line under the tiles, labelled as your session’s and kept in your own browser — because this demo has no database, and a figure a visitor can change is not the institution’s measurement.Your session’s recount. The figures above are the record: they are counted from dispositions two named officers wrote, and nothing you do on this page moves them. Mark a row and a second pair appears here — the record’s verdicts with your marks laid over them, recomputed as you go. Your marks are kept in this browser and stamped with the time you made them, so they are still here when you come back; they reach no server and no other reader, because this demo has no database.
Precision. Of the detections in the adjudicated sample, the share a named officer marked a true positive. It describes the flags this control raised, over the sample stated in the sampling rule — not the control’s behaviour on traffic it never flagged.
False-positive rate of flagged-and-reviewed. The complement of precision over the same sample and the same denominator: the share of reviewed flags marked wrong. It is not a false-positive rate over the negative population, which would need a count of unflagged submissions.
Recall — sensitive submissions detection missed. No figure on any surface measures it, and none can be derived from the adjudicated sample: adjudication only ever sees events the engine already flagged. It would take: Ground truth independent of the detector — a second, differently-built classifier over the same traffic, or a human review of a sample of unflagged interactions.
Coverage — the share of staff and devices the sensor reaches. The pilot has no deployment inventory in this record, so a detection count has no population behind it. It would take: An endpoint inventory reconciled against the staff directory, per week.
The adjudication queue is a stratified sample of the detections in the trailing thirty days: each data class contributes in proportion to its share of the window, and within a class events are taken in a fixed order derived from the pilot seed. Detections separately reported as errors are included as well.
Reported errors are in the sample by construction rather than by the draw, so precision over this sample is a floor on the sample’s own accuracy and not an unbiased estimate of the control’s. Recall is not measured at all — see the accuracy record.
Who marked itDraws alternate between the two adjudicators in the sampled order, so neither officer marks a contiguous block and the sample carries both signatures.
M. Halloran: Owns the control and proposes the policy versions. Marks the odd-numbered draws. Y. Abadi: Independent reviewer: does not propose policy and does not own the console. Marks the even-numbered draws.
16 detections nobody has looked at yet
The next draw, by the same rule as the 120 below — which is the point of drawing it the same way: a precision figure this quarter is only comparable with next quarter’s if the two samples were taken alike. These carry no verdict, and the figures above are computed without them, because an unreviewed detection is not evidence of anything.16 waiting, none marked yet. Your marks stay in this browser, stamped with the time you made them, and are still here on your next visit. They are yours alone: the record above is counted from two named officers’ dispositions, this queue is redrawn from the seeded log rather than from what you marked, and nothing here is sent anywhere. What a deployment does instead — write a disposition to the event, drop it from tomorrow’s draw, and put it in the examiner’s pack — needs the database this tier does not have.
| Excerpt | Classes | Decision | Your verdict | |
|---|---|---|---|---|
| What does a returned item on account 00408779 do to the overdraft fee schedule?B. Mwangi, Finance · Jul 17, 2026 at 9:34 AM | Account Numbers | Monitorv3 | Chain | |
| Reconcile these two entries against deposit account 00863164 and tell me which one is the duplicate posting.L. Tran, Retail & Branch Ops · Jul 23, 2026 at 4:55 PM | Account Numbers | Monitorv4 | Chain | |
| Customer Espen Mackintosh is disputing three ACH debits on account 00849552. Draft a response that explains the provisional credit timeline.N. Oyelaran, Credit Administration · Jul 27, 2026 at 10:16 AM | Member PII, Account Numbers | Warnv4 | Chain | |
| Draft the recommendation section of a credit memorandum for Andre Asante, requested facility $518,612, secured by equipment.N. Oyelaran, Credit Administration · Jul 29, 2026 at 3:09 PM | Credit Memos, Member PII | Warnv4 | Chain | |
| Rewrite the mitigants paragraph in the attached credit memorandum for Alicia Qureshi at Zanesville Enterprises, requested $1,313,374. Keep the committee language, drop the hedging.N. Oyelaran, Credit Administration · Jul 29, 2026 at 3:52 PM | Credit Memos, Member PII | Warnv4 | Chain | |
| This call keeps returning 401. The header is Authorization: Bearer ghp_aym2myuy2yuiqeuuuqaum2eyyiua — what am I missing?G. Iyer, IT · Jul 30, 2026 at 8:19 AM | API Keys / Creds | Blockv4 | Chain | |
| What does covenant 6(b) in loan CL-2026-50577 actually require at each quarter end?T. Beaulieu, Commercial Lending · Aug 3, 2026 at 8:41 AM | Loan Files | Monitorv5 | Chain | |
| Write a follow-up note to Camille Nilsen at (419) 555-0149 about the certificate of deposit maturing next month.A. Bergström, Credit Administration · Aug 4, 2026 at 4:15 PM | Member PII | Warnv5 | Chain | |
| What is the collateral position on loan CL-2026-06494? The borrower is Marisol Krishnamurthy, and the appraisal came in at $2,180,680.R. Okafor, Commercial Lending · Aug 4, 2026 at 5:54 PM | Loan Files, Member PII | Redactv5 | Chain | |
| Summarize this complaint. Customer: Greta Nakashima, callback (419) 555-0103, branch: Alder Creek Main.H. Delacroix, Credit Administration · Aug 6, 2026 at 4:20 PM | Member PII | Warnv5 | Chain | |
| How should I phrase a beneficiary change confirmation for Nikhil Ramanathan, reachable on (419) 555-0182?W. Nyathi, Compliance/BSA · Aug 7, 2026 at 1:17 PM | Member PII | Monitorv5 | Chain | |
| Reconcile these two entries against deposit account 00117740 and tell me which one is the duplicate posting.P. Nakamura, Retail & Branch Ops · Aug 7, 2026 at 4:43 PM | Account Numbers | Monitorv5 | Chain | |
| Please review this loan application for Wren Callahan, SSN 453-56-4914, account #00196680. The debt-to-income ratio is 49% — is that inside our commercial guideline?D. Whitfield, Commercial Lending · Aug 11, 2026 at 8:25 AM | Member PII, SSN / Tax ID, Account Numbers | Redactv5 | Chain | |
| Summarize the credit file for loan CL-2026-68449 — borrower Bela Nilsen, guarantor on the Kettering-Freight Systems relationship. Focus on covenant history and past-due notes.A. Bergström, Credit Administration · Aug 11, 2026 at 2:38 PM | Loan Files, Member PII | Warnv5 | Chain | |
| What is the collateral position on loan CL-2026-52110? The borrower is Marisol Krishnamurthy, and the appraisal came in at $807,448.A. Bergström, Credit Administration · Aug 14, 2026 at 9:42 AM | Loan Files, Member PII | Warnv5 | Chain | |
| The card ending in this number keeps declining: 4539860236701693. The customer is Imani Kovalenko. What should the branch check first?A. Bergström, Credit Administration · Aug 14, 2026 at 4:26 PM | Account Numbers, Member PII | Warnv5 | Chain |
A codename in the institution’s dictionary, firing on a landscaping quote
“Project Sycamore” is an M&A codename at Alder Creek Community Bank and a tree everywhere else. The bank chose the word, the dictionary detector found it exactly where the bank said to look, and the detection is still wrong — which is what a custom dictionary costs and why it is configuration rather than code.Draft a polite reply to the landscaping vendor about the Project Sycamore planting schedule — the Main Street beds are behind and we want them finished before the branch anniversary.
proved by a checksum or a format rule · matched by a pattern, which is a shape, or by the taxonomy or the institution’s dictionary, which are vocabulary markers — neither is a proof
It is one of the 8 errors in the 120-event sample, and it is in the denominator of the 6.7% above rather than excused out of it. Open its chain
120 events adjudicated
The worked half of the same draw: the two figures at the top are counted from these verdicts and from nothing else. Read the excerpt, read the spans, mark it. Your marks are kept in your own browser, stamped with the time you made them, so the work you did here is still here next time — this demo has no database, so they reach no server and no other reader, and the queue says which of the two it is rather than showing a control that quietly saves nothing.| Excerpt | Classes | Decision | Recorded verdict | Your verdict | |
|---|---|---|---|---|---|
| Verify this routing number before I release the wire: 040683856. It came in on a faxed instruction.S. Adeyemi, Retail & Branch Ops · Aug 7, 2026 at 12:28 PM | Account Numbers | Monitorv5 | True positiveY. Abadi | Chain | |
| Draft the recommendation section of a credit memorandum for Lucian Benali, requested facility $843,784, secured by equipment.H. Delacroix, Credit Administration · Aug 7, 2026 at 11:01 AM | Credit Memos, Member PII | Warnv5 | True positiveM. Halloran | Chain | |
| What is the collateral position on loan CL-2026-10818? The borrower is Tomas Krishnamurthy, and the appraisal came in at $1,264,100.A. Bergström, Credit Administration · Aug 7, 2026 at 9:20 AM | Loan Files, Member PII | Warnv5 | True positiveM. Halloran | Chain | |
| Summarize the credit file for loan CL-2026-88917 — borrower Camille Nilsen, guarantor on the Hocking-Hills Manufacturing relationship. Focus on covenant history and past-due notes.M. Ramaswamy, Commercial Lending · Aug 7, 2026 at 8:59 AM | Loan Files, Member PII | Redactv5 | True positiveM. Halloran | Chain | |
| Pull the payment history out of the file for loan CL-2026-69271 and tell me whether Halima Nakashima has been late more than twice in the last two years.M. Ramaswamy, Commercial Lending · Aug 7, 2026 at 8:40 AM | Loan Files, Member PII | Redactv5 | True positiveM. Halloran | Chain | |
| Summarize the credit file for loan CL-2026-86645 — borrower Wren Mackintosh, guarantor on the Chagrin-Valley Properties relationship. Focus on covenant history and past-due notes.M. Ramaswamy, Commercial Lending · Aug 7, 2026 at 8:39 AM | Loan Files, Member PII | Redactv5 | True positiveM. Halloran | Chain | |
| Tighten this press release quote: “Alder Creek is proud to sponsor the county fair,” said E. Sandoval. Keep the published routing number 041502211 in the donation footer.E. Sandoval, Executive · Aug 6, 2026 at 4:40 PM | Member PII, Account Numbers | Monitorv5 | False positiveM. Halloran | Chain | |
| Draft a birthday message for Andre Achterberg, date of birth 1985-09-26, who has banked with us for eleven years.A. Bergström, Credit Administration · Aug 6, 2026 at 3:11 PM | Member PII | Warnv5 | True positiveY. Abadi | Chain | |
| Please review this loan application for Simone Callahan, SSN 455-77-3946, account #00186380. The debt-to-income ratio is 37% — is that inside our commercial guideline?M. Ramaswamy, Commercial Lending · Aug 6, 2026 at 2:48 PM | Member PII, SSN / Tax ID, Account Numbers | Redactv5 | True positiveY. Abadi | Chain | |
| Write a follow-up note to Fiona Pemberton at (419) 555-0121 about the certificate of deposit maturing next month.D. Whitfield, Commercial Lending · Aug 6, 2026 at 2:26 PM | Member PII | Redactv5 | True positiveY. Abadi | Chain | |
| I am preparing an annual review on loan CL-2026-57692. Borrower Jamal Beauchamp, operating company Maumee-River Holdings. What questions should the committee expect?R. Okafor, Commercial Lending · Aug 6, 2026 at 1:11 PM | Loan Files, Member PII | Redactv5 | True positiveY. Abadi | Chain | |
| Verify this routing number before I release the wire: 047238615. It came in on a faxed instruction.D. Whitfield, Commercial Lending · Aug 6, 2026 at 10:59 AM | Account Numbers | Tokenizev5 | True positiveY. Abadi | Chain | |
| Summarize this complaint. Customer: Greta Vasquez, callback (419) 555-0151, branch: Alder Creek Main.M. Halloran, Compliance/BSA · Aug 6, 2026 at 8:30 AM | Member PII | Monitorv5 | True positiveY. Abadi | Chain | |
| Can you sanity-check the income calculation? Borrower Clemence Thibodeaux, SSN 422-07-9321, account 00136617, gross monthly $15,994, two years of Schedule C attached.D. Whitfield, Commercial Lending · Aug 5, 2026 at 5:45 PM | Member PII, SSN / Tax ID, Account Numbers | Redactv5 | True positiveM. Halloran | Chain | |
| Compare the borrowing base certificate filed under loan CL-2026-22835 against the advance rate in the commitment letter for Chagrin-Valley Properties.A. Bergström, Credit Administration · Aug 5, 2026 at 4:46 PM | Loan Files | Monitorv5 | True positiveY. Abadi | Chain | |
| Draft the recommendation section of a credit memorandum for Priya Chaudhary, requested facility $278,296, secured by equipment.R. Okafor, Commercial Lending · Aug 5, 2026 at 3:15 PM | Credit Memos, Member PII | Redactv5 | True positiveM. Halloran | Chain | |
| Pull the payment history out of the file for loan CL-2026-80311 and tell me whether Ingrid Vasquez has been late more than twice in the last two years.D. Whitfield, Commercial Lending · Aug 5, 2026 at 12:54 PM | Loan Files, Member PII | Redactv5 | True positiveM. Halloran | Chain | |
| Summarize this complaint. Customer: Devrim Whitcomb, callback (419) 555-0159, branch: Alder Creek Main.T. Beaulieu, Commercial Lending · Aug 5, 2026 at 12:42 PM | Member PII | Redactv5 | True positiveY. Abadi | Chain | |
| What does a returned item on account 00225131 do to the overdraft fee schedule?I. Quintero, Retail & Branch Ops · Aug 5, 2026 at 11:49 AM | Account Numbers | Monitorv5 | True positiveY. Abadi | Chain | |
| Confirm the wire instructions for Soren Beauchamp — routing 042907886, account 00952221. Draft the callback script.D. Whitfield, Commercial Lending · Aug 5, 2026 at 11:18 AM | Member PII, Account Numbers | Redactv5 | True positiveM. Halloran | Chain | |
| Confirm the wire instructions for Kwame Eriksen — routing 043164806, account 00857561. Draft the callback script.R. Okafor, Commercial Lending · Aug 5, 2026 at 9:49 AM | Member PII, Account Numbers | Redactv5 | True positiveY. Abadi | Chain | |
| Turn the covenant table in the Kettering-Freight Systems credit memorandum into plain sentences a relationship manager can read out. Fixed-charge coverage is 1.17x.N. Oyelaran, Credit Administration · Aug 4, 2026 at 4:10 PM | Credit Memos | Monitorv5 | True positiveY. Abadi | Chain | |
| The facilities contract carries note number CL-2026-00912 — that is the vendor’s document number, not one of ours. How should I file it?P. Nakamura, Retail & Branch Ops · Aug 4, 2026 at 1:05 PM | Loan Files | Monitorv5 | False positiveY. Abadi | Chain | |
| Confirm the wire instructions for Marcus Brandt — routing 043579886, account 00084061. Draft the callback script.L. Tran, Retail & Branch Ops · Aug 4, 2026 at 11:42 AM | Member PII, Account Numbers | Monitorv5 | True positiveY. Abadi | Chain | |
| How should I phrase a beneficiary change confirmation for Soren Obradovic, reachable on (419) 555-0198?J. Kowalczyk, Retail & Branch Ops · Aug 4, 2026 at 11:38 AM | Member PII | Monitorv5 | True positiveY. Abadi | Chain | |
| What is the collateral position on loan CL-2026-32358? The borrower is Nadira Hollingsworth, and the appraisal came in at $960,160.K. Osei, Commercial Lending · Aug 4, 2026 at 9:26 AM | Loan Files, Member PII | Redactv5 | True positiveM. Halloran | Chain | |
| Summarize the borrower file below for underwriting. The applicant is Hassan Achterberg, SSN 294-75-9315, deposit account 00696423, requested amount $257,020.H. Delacroix, Credit Administration · Aug 3, 2026 at 5:36 PM | Member PII, SSN / Tax ID, Account Numbers | Warnv5 | True positiveY. Abadi | Chain | |
| This call keeps returning 401. The header is Authorization: Bearer sk-live-eyaqyeqm6mm6q6iqaeyeuuyyaeey — what am I missing?V. Sokolov, IT · Aug 3, 2026 at 4:03 PM | API Keys / Creds | Blockv5 | True positiveM. Halloran | Chain | |
| What is the collateral position on loan CL-2026-42838? The borrower is Gunnar Krishnamurthy, and the appraisal came in at $1,916,544.M. Ramaswamy, Commercial Lending · Aug 3, 2026 at 3:19 PM | Loan Files, Member PII | Redactv5 | True positiveM. Halloran | Chain | |
| The card ending in this number keeps declining: 4539996510301709. The customer is Lena Okonjo. What should the branch check first?I. Quintero, Retail & Branch Ops · Aug 3, 2026 at 10:33 AM | Account Numbers, Member PII | Monitorv5 | True positiveM. Halloran | Chain |