FastPay AI: Autonomous 3-Way Invoice Matching Agent

3-Way Matching: thousands of manual hours in every Accounts Payable (AP) team

Every Accounts Payable (AP) team runs the same gauntlet daily: for every vendor invoice received, a clerk must manually pull the corresponding Purchase Order and Warehouse receipt, then verify that quantities, prices, line items, and totals agree across all three documents. Any mismatch, a shortage, a price increase, an unauthorized SKU, a duplicate submission, must be caught before payment is released.

This is 3-Way Matching. It is repetitive, time-consuming, and deeply error-prone at scale. It is also a textbook agentic AI use case: structured document comparison, tool-based data lookup, deterministic rules, and a final reasoning step that must produce an auditable decision.

Custom orchestrator: no framework, full control

The agent is built as a hand-rolled orchestrator in TypeScript, no LangChain, no LangGraph. This was a deliberate choice: 3-Way Matching has a fixed, deterministic pipeline. The steps never change order, there is no dynamic tool routing, and the control flow is simple enough that a framework would add overhead without value.

// orchestrator.ts, simplified
export async function runAgent(invoiceId: string, emit: EmitFn): Promise {
  const invoice  = await getInvoiceById(invoiceId)
  await checkDuplicate(invoice.invoice_number)   // fast, no API

  const extracted   = await extractPdf(invoice.pdf_path)  // Gemini Vision
  const po          = await lookupPo(extracted.po_reference)
  const wms         = await queryWms(po.id)
  const vendorMatch = await fuzzyMatchVendor(extracted.vendor_name, po.vendor_name)
  const fxResult    = await convertCurrency(...)           // if currencies differ

  const decision = await reasonAndDecide({ extracted, po, wms, vendorMatch, fxResult })
  emit({ step: 'decide', status: 'done', detail: decision.status })

  await saveResult(invoiceId, decision)
  return buildResult(invoiceId, decision, trace)
}

Reading invoices the way a human would: including messy ones

The first tool call is the hardest: turning an unstructured invoice image into a structured JSON object. The agent sends the raw PDF or image file to Gemini 2.5 Flash as base64-encoded inline data, along with a schema-anchored extraction prompt. The response is validated with a Zod schema before any downstream tool sees it.

// extract-pdf.ts
const result = await model.generateContent([
  EXTRACTION_PROMPT,                         // structured output instructions
  { inlineData: { mimeType, data: base64 } } // raw file bytes
])

// Zod validates the response, rejects any hallucinated fields
return ExtractedInvoiceSchema.parse(JSON.parse(result.response.text()))

The 12 test scenarios cover 5 invoice variants: clean digital PDF, scanned document, phone photo, handwritten annotation, and crumpled/degraded. All 5 formats are tested against the same extraction schema to prove robustness.

Five typed tools that query real data sources

After extraction, the orchestrator runs a sequence of typed tool calls. Each tool is a pure async function, no side effects, no shared state, easy to test in isolation.

Deterministic rules + LLM-generated explanation

The final reason_and_decide step combines two things: a deterministic rule engine that catches every known mismatch type, and a Gemini call that writes a plain-English explanation of exactly what was found and why.

Watching the agent think: in real time

The UI is a Next.js 14 App Router application. When a user clicks an invoice card, the browser opens a Server-Sent Events connection to /api/agent/stream. Every tool call in the orchestrator emits a TraceEvent that is pushed down the stream immediately, the trace panel updates step-by-step as the agent works.

// SSE stream consumer, client side
const reader = res.body.getReader()
while (true) {
  const { done, value } = await reader.read()
  if (done) break

  const events = parseSSE(value)
  for (const event of events) {
    if (event.type === 'step')   applyStep(event)  // updates trace panel
    if (event.type === 'result') setDecision(event) // reveals decision
  }
}

Proving the agent is reliable: with numbers

Every AI demo should have an eval suite. This one runs automatically: hit "Run Eval" and the agent processes all 12 scenarios fresh, then computes a full report against ground-truth labels defined in data/scenarios.json.

ERP-style data visibility baked into the demo

One of the goals was to make the data behind the agent visible, to show that this isn't a black box but a system you could wire into a real supply chain. Two dedicated views provide that transparency.

The impacted-SKU logic is deterministic: for PRICE_MISMATCH, it compares invoice unit price vs PO price per SKU; for SHORTAGE, invoice qty vs WMS received qty; for UNAUTHORIZED_ITEMS, SKUs present in the invoice but absent from the PO.

Upload any real invoice and watch the agent find a flaw

Beyond the 12 pre-seeded scenarios, users can upload any invoice PDF, JPEG, PNG, or WEBP. The system:

• Extracts the invoice data with Gemini Vision
• Generates a synthetic PO and WMS receipt from the extracted data
• Injects one intentional discrepancy (randomly chosen: price variance, quantity shortage, or unauthorized line item)
• Runs the full agent pipeline and explains what it found

Uploaded files are written to /tmp (Vercel's only writable directory) and deleted immediately after processing. They are never committed to git or persisted beyond the request.

Production-grade on a $0/month budget