93.3% precision, 6.7% false-positive rate
Counted from the recorded dispositions of the sample below and from nothing else — the verdicts two named officers wrote, each quoted over the same denominator, stated beside it. Your own marks do not move these two. They are laid over the record and recounted on the line under the tiles, labelled as your session’s and kept in your own browser — because this demo has no database, and a figure a visitor can change is not the institution’s measurement.Your session’s recount. The figures above are the record: they are counted from dispositions two named officers wrote, and nothing you do on this page moves them. Mark a row and a second pair appears here — the record’s verdicts with your marks laid over them, recomputed as you go. Your marks are kept in this browser and stamped with the time you made them, so they are still here when you come back; they reach no server and no other reader, because this demo has no database.
Precision. Of the detections in the adjudicated sample, the share a named officer marked a true positive. It describes the flags this control raised, over the sample stated in the sampling rule — not the control’s behaviour on traffic it never flagged.
False-positive rate of flagged-and-reviewed. The complement of precision over the same sample and the same denominator: the share of reviewed flags marked wrong. It is not a false-positive rate over the negative population, which would need a count of unflagged submissions.
Recall — sensitive submissions detection missed. No figure on any surface measures it, and none can be derived from the adjudicated sample: adjudication only ever sees events the engine already flagged. It would take: Ground truth independent of the detector — a second, differently-built classifier over the same traffic, or a human review of a sample of unflagged interactions.
Coverage — the share of staff and devices the sensor reaches. The pilot has no deployment inventory in this record, so a detection count has no population behind it. It would take: An endpoint inventory reconciled against the staff directory, per week.
The adjudication queue is a stratified sample of the detections in the trailing thirty days: each data class contributes in proportion to its share of the window, and within a class events are taken in a fixed order derived from the pilot seed. Detections separately reported as errors are included as well.
Reported errors are in the sample by construction rather than by the draw, so precision over this sample is a floor on the sample’s own accuracy and not an unbiased estimate of the control’s. Recall is not measured at all — see the accuracy record.
Who marked itDraws alternate between the two adjudicators in the sampled order, so neither officer marks a contiguous block and the sample carries both signatures.
M. Halloran: Owns the control and proposes the policy versions. Marks the odd-numbered draws. Y. Abadi: Independent reviewer: does not propose policy and does not own the console. Marks the even-numbered draws.
16 detections nobody has looked at yet
The next draw, by the same rule as the 120 below — which is the point of drawing it the same way: a precision figure this quarter is only comparable with next quarter’s if the two samples were taken alike. These carry no verdict, and the figures above are computed without them, because an unreviewed detection is not evidence of anything.16 waiting, none marked yet. Your marks stay in this browser, stamped with the time you made them, and are still here on your next visit. They are yours alone: the record above is counted from two named officers’ dispositions, this queue is redrawn from the seeded log rather than from what you marked, and nothing here is sent anywhere. What a deployment does instead — write a disposition to the event, drop it from tomorrow’s draw, and put it in the examiner’s pack — needs the database this tier does not have.
| Excerpt | Classes | Decision | Your verdict | |
|---|---|---|---|---|
| What does a returned item on account 00408779 do to the overdraft fee schedule?B. Mwangi, Finance · Jul 17, 2026 at 9:34 AM | Account Numbers | Monitorv3 | Chain | |
| Reconcile these two entries against deposit account 00863164 and tell me which one is the duplicate posting.L. Tran, Retail & Branch Ops · Jul 23, 2026 at 4:55 PM | Account Numbers | Monitorv4 | Chain | |
| Customer Espen Mackintosh is disputing three ACH debits on account 00849552. Draft a response that explains the provisional credit timeline.N. Oyelaran, Credit Administration · Jul 27, 2026 at 10:16 AM | Member PII, Account Numbers | Warnv4 | Chain | |
| Draft the recommendation section of a credit memorandum for Andre Asante, requested facility $518,612, secured by equipment.N. Oyelaran, Credit Administration · Jul 29, 2026 at 3:09 PM | Credit Memos, Member PII | Warnv4 | Chain | |
| Rewrite the mitigants paragraph in the attached credit memorandum for Alicia Qureshi at Zanesville Enterprises, requested $1,313,374. Keep the committee language, drop the hedging.N. Oyelaran, Credit Administration · Jul 29, 2026 at 3:52 PM | Credit Memos, Member PII | Warnv4 | Chain | |
| This call keeps returning 401. The header is Authorization: Bearer ghp_aym2myuy2yuiqeuuuqaum2eyyiua — what am I missing?G. Iyer, IT · Jul 30, 2026 at 8:19 AM | API Keys / Creds | Blockv4 | Chain | |
| What does covenant 6(b) in loan CL-2026-50577 actually require at each quarter end?T. Beaulieu, Commercial Lending · Aug 3, 2026 at 8:41 AM | Loan Files | Monitorv5 | Chain | |
| Write a follow-up note to Camille Nilsen at (419) 555-0149 about the certificate of deposit maturing next month.A. Bergström, Credit Administration · Aug 4, 2026 at 4:15 PM | Member PII | Warnv5 | Chain | |
| What is the collateral position on loan CL-2026-06494? The borrower is Marisol Krishnamurthy, and the appraisal came in at $2,180,680.R. Okafor, Commercial Lending · Aug 4, 2026 at 5:54 PM | Loan Files, Member PII | Redactv5 | Chain | |
| Summarize this complaint. Customer: Greta Nakashima, callback (419) 555-0103, branch: Alder Creek Main.H. Delacroix, Credit Administration · Aug 6, 2026 at 4:20 PM | Member PII | Warnv5 | Chain | |
| How should I phrase a beneficiary change confirmation for Nikhil Ramanathan, reachable on (419) 555-0182?W. Nyathi, Compliance/BSA · Aug 7, 2026 at 1:17 PM | Member PII | Monitorv5 | Chain | |
| Reconcile these two entries against deposit account 00117740 and tell me which one is the duplicate posting.P. Nakamura, Retail & Branch Ops · Aug 7, 2026 at 4:43 PM | Account Numbers | Monitorv5 | Chain | |
| Please review this loan application for Wren Callahan, SSN 453-56-4914, account #00196680. The debt-to-income ratio is 49% — is that inside our commercial guideline?D. Whitfield, Commercial Lending · Aug 11, 2026 at 8:25 AM | Member PII, SSN / Tax ID, Account Numbers | Redactv5 | Chain | |
| Summarize the credit file for loan CL-2026-68449 — borrower Bela Nilsen, guarantor on the Kettering-Freight Systems relationship. Focus on covenant history and past-due notes.A. Bergström, Credit Administration · Aug 11, 2026 at 2:38 PM | Loan Files, Member PII | Warnv5 | Chain | |
| What is the collateral position on loan CL-2026-52110? The borrower is Marisol Krishnamurthy, and the appraisal came in at $807,448.A. Bergström, Credit Administration · Aug 14, 2026 at 9:42 AM | Loan Files, Member PII | Warnv5 | Chain | |
| The card ending in this number keeps declining: 4539860236701693. The customer is Imani Kovalenko. What should the branch check first?A. Bergström, Credit Administration · Aug 14, 2026 at 4:26 PM | Account Numbers, Member PII | Warnv5 | Chain |
A codename in the institution’s dictionary, firing on a landscaping quote
“Project Sycamore” is an M&A codename at Alder Creek Community Bank and a tree everywhere else. The bank chose the word, the dictionary detector found it exactly where the bank said to look, and the detection is still wrong — which is what a custom dictionary costs and why it is configuration rather than code.Draft a polite reply to the landscaping vendor about the Project Sycamore planting schedule — the Main Street beds are behind and we want them finished before the branch anniversary.
proved by a checksum or a format rule · matched by a pattern, which is a shape, or by the taxonomy or the institution’s dictionary, which are vocabulary markers — neither is a proof
It is one of the 8 errors in the 120-event sample, and it is in the denominator of the 6.7% above rather than excused out of it. Open its chain
120 events adjudicated
The worked half of the same draw: the two figures at the top are counted from these verdicts and from nothing else. Read the excerpt, read the spans, mark it. Your marks are kept in your own browser, stamped with the time you made them, so the work you did here is still here next time — this demo has no database, so they reach no server and no other reader, and the queue says which of the two it is rather than showing a control that quietly saves nothing.| Excerpt | Classes | Decision | Recorded verdict | Your verdict | |
|---|---|---|---|---|---|
| Draft a birthday message for Priya Krishnamurthy, date of birth 1991-04-17, who has banked with us for eleven years.A. Bergström, Credit Administration · Jul 23, 2026 at 1:04 PM | Member PII | Warnv4 | True positiveY. Abadi | Chain | |
| Tighten the risk-rating rationale in this credit memorandum so it reads plainly for the loan committee. Borrower is Yusuf Nilsen, Hocking-Hills Manufacturing.T. Beaulieu, Commercial Lending · Jul 23, 2026 at 12:51 PM | Credit Memos, Member PII | Warnv4 | True positiveY. Abadi | Chain | |
| Write a hold notice for account 00014111, held by Emiliano Fontenot. Deposit is a business check over the case-by-case threshold.B. Mwangi, Finance · Jul 23, 2026 at 11:47 AM | Account Numbers, Member PII | Monitorv4 | True positiveY. Abadi | Chain | |
| What does a returned item on account 00737427 do to the overdraft fee schedule?C. Ferrante, Finance · Jul 22, 2026 at 5:10 PM | Account Numbers | Monitorv4 | True positiveY. Abadi | Chain | |
| I am preparing an annual review on loan CL-2026-71292. Borrower Caleb Obradovic, operating company Wapakoneta-Auto Group. What questions should the committee expect?N. Oyelaran, Credit Administration · Jul 22, 2026 at 3:54 PM | Loan Files, Member PII | Warnv4 | True positiveM. Halloran | Chain | |
| Write a hold notice for account 00205627, held by Tobias Benali. Deposit is a business check over the case-by-case threshold.E. Lindqvist, Finance · Jul 22, 2026 at 1:28 PM | Account Numbers, Member PII | Monitorv4 | True positiveM. Halloran | Chain | |
| Confirm the wire instructions for Jamal Villareal — routing 042990961, account 00114945. Draft the callback script.K. Osei, Commercial Lending · Jul 22, 2026 at 9:25 AM | Member PII, Account Numbers | Warnv4 | True positiveY. Abadi | Chain | |
| How should I phrase a beneficiary change confirmation for Anouk Brandt, reachable on (419) 555-0142?J. Kowalczyk, Retail & Branch Ops · Jul 22, 2026 at 9:09 AM | Member PII | Monitorv4 | True positiveY. Abadi | Chain | |
| Pull the payment history out of the file for loan CL-2026-69299 and tell me whether Devrim Nakashima has been late more than twice in the last two years.T. Beaulieu, Commercial Lending · Jul 21, 2026 at 10:33 AM | Loan Files, Member PII | Warnv4 | True positiveM. Halloran | Chain | |
| What should a training module on writing a credit memorandum cover for new analysts? This is for the curriculum, not a live file.Y. Abadi, Compliance/BSA · Jul 21, 2026 at 10:12 AM | Credit Memos | Monitorv4 | False positiveY. Abadi | Chain | |
| Explain the settlement timing for this card: 4539144022401804. The merchant says it authorized on Friday.B. Mwangi, Finance · Jul 21, 2026 at 9:29 AM | Account Numbers | Monitorv4 | True positiveY. Abadi | Chain | |
| Explain the settlement timing for this card: 4539850014800916. The merchant says it authorized on Friday.C. Ferrante, Finance · Jul 20, 2026 at 3:36 PM | Account Numbers | Monitorv4 | True positiveM. Halloran | Chain | |
| Confirm the wire instructions for Soren Villareal — routing 045221129, account 00857817. Draft the callback script.P. Nakamura, Retail & Branch Ops · Jul 20, 2026 at 11:04 AM | Member PII, Account Numbers | Monitorv4 | True positiveY. Abadi | Chain | |
| Write a follow-up note to Deirdre Winterbourne at (419) 555-0125 about the certificate of deposit maturing next month.L. Tran, Retail & Branch Ops · Jul 20, 2026 at 10:48 AM | Member PII | Monitorv4 | True positiveY. Abadi | Chain | |
| Customer Bela Pemberton is disputing three ACH debits on account 00898352. Draft a response that explains the provisional credit timeline.H. Delacroix, Credit Administration · Jul 20, 2026 at 10:16 AM | Member PII, Account Numbers | Warnv4 | True positiveM. Halloran | Chain | |
| Draft the recommendation section of a credit memorandum for Hassan Krishnamurthy, requested facility $1,591,768, secured by equipment.A. Bergström, Credit Administration · Jul 20, 2026 at 9:38 AM | Credit Memos, Member PII | Warnv4 | True positiveM. Halloran | Chain | |
| Pull the payment history out of the file for loan CL-2026-01699 and tell me whether Lena Whitcomb has been late more than twice in the last two years.A. Bergström, Credit Administration · Jul 20, 2026 at 8:22 AM | Loan Files, Member PII | Warnv4 | True positiveY. Abadi | Chain | |
| How should I phrase a beneficiary change confirmation for Nikhil Ekwueme, reachable on (419) 555-0182?N. Oyelaran, Credit Administration · Jul 17, 2026 at 5:57 PM | Member PII | Warnv3 | True positiveY. Abadi | Chain | |
| Draft a decline letter for Ingrid Solberg (SSN 143-57-8743, primary account 00228174). Reason: insufficient operating cash flow over the trailing twelve months.T. Beaulieu, Commercial Lending · Jul 17, 2026 at 4:44 PM | Member PII, SSN / Tax ID, Account Numbers | Warnv3 | True positiveY. Abadi | Chain | |
| Write a follow-up note to Astrid Toth at (419) 555-0105 about the certificate of deposit maturing next month.P. Nakamura, Retail & Branch Ops · Jul 17, 2026 at 2:12 PM | Member PII | Monitorv3 | True positiveY. Abadi | Chain | |
| Write a hold notice for account 00253171, held by Tobias Chaudhary. Deposit is a business check over the case-by-case threshold.I. Quintero, Retail & Branch Ops · Jul 17, 2026 at 1:24 PM | Account Numbers, Member PII | Monitorv3 | True positiveM. Halloran | Chain | |
| Pull the payment history out of the file for loan CL-2026-80919 and tell me whether Meredith Kovalenko has been late more than twice in the last two years.D. Whitfield, Commercial Lending · Jul 17, 2026 at 12:42 PM | Loan Files, Member PII | Warnv3 | True positiveM. Halloran | Chain | |
| Verify this routing number before I release the wire: 047155497. It came in on a faxed instruction.J. Kowalczyk, Retail & Branch Ops · Jul 17, 2026 at 12:38 PM | Account Numbers | Monitorv3 | True positiveM. Halloran | Chain | |
| Does backup withholding apply if the W-9 shows 637-94-5442 and the name control does not match?L. Tran, Retail & Branch Ops · Jul 17, 2026 at 10:29 AM | SSN / Tax ID | Monitorv3 | True positiveM. Halloran | Chain | |
| Write a follow-up note to Bela Nilsen at (419) 555-0153 about the certificate of deposit maturing next month.K. Osei, Commercial Lending · Jul 16, 2026 at 4:31 PM | Member PII | Warnv3 | True positiveY. Abadi | Chain | |
| Reconcile these two entries against deposit account 00531512 and tell me which one is the duplicate posting.B. Mwangi, Finance · Jul 16, 2026 at 2:58 PM | Account Numbers | Monitorv3 | True positiveM. Halloran | Chain | |
| Verify this routing number before I release the wire: 044748739. It came in on a faxed instruction.P. Nakamura, Retail & Branch Ops · Jul 16, 2026 at 2:07 PM | Account Numbers | Monitorv3 | True positiveY. Abadi | Chain | |
| Customer Beatriz Aguilar is disputing three ACH debits on account 00911964. Draft a response that explains the provisional credit timeline.A. Bergström, Credit Administration · Jul 16, 2026 at 10:46 AM | Member PII, Account Numbers | Warnv3 | True positiveM. Halloran | Chain | |
| Rewrite the mitigants paragraph in the attached credit memorandum for Rashid Obradovic at Sandusky-Millwork LLC, requested $1,066,074. Keep the committee language, drop the hedging.T. Beaulieu, Commercial Lending · Jul 16, 2026 at 9:49 AM | Credit Memos, Member PII | Warnv3 | True positiveY. Abadi | Chain | |
| Summarize the credit file for loan CL-2026-87017 — borrower Camille Prieto, guarantor on the Chagrin-Valley Properties relationship. Focus on covenant history and past-due notes.M. Ramaswamy, Commercial Lending · Jul 16, 2026 at 8:25 AM | Loan Files, Member PII | Warnv3 | True positiveM. Halloran | Chain |