Architecture

System Overview

Receipt OCR App (Next.js, deployed as a Docker image)
    ├── Storage Brain SDK               → object storage (file uploads)
    ├── Google Cloud Vision             → OCR            (text extraction)
    ├── OpenRouter                      → LLM            (classification + chat)
    └── @marlinjai/data-table-adapter-prisma → PostgreSQL (structured data)

Stack note. This app ran on Cloudflare Workers with D1 and @opennextjs/cloudflare earlier in its life. That is retired: persistence is PostgreSQL through Prisma, and the app ships as a Docker image. This document described the old stack long after the move, which is corrected here. The only surviving Cloudflare dependency is the DOCS site, deployed to Cloudflare Pages by .github/workflows/deploy-docs.yml.

┌──────────────────────────────────────────────────────────────────────┐
│                 Receipt OCR App (Next.js, Docker image)             │
├──────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ┌────────────┐  ┌──────────────┐  ┌──────────────┐  ┌───────────┐  │
│  │  Upload    │  │  Dashboard   │  │  AI Chat     │  │   API     │  │
│  │  Page      │  │  (4 views)   │  │  Sidebar     │  │  Routes   │  │
│  └─────┬──────┘  └──────┬───────┘  └──────┬───────┘  └─────┬─────┘  │
│        │                │                  │                │        │
│        ▼                ▼                  ▼                ▼        │
│  ┌───────────────────────────────────────────────────────────────┐   │
│  │                     Component Layer                           │   │
│  │  ┌──────────────┐ ┌─────────────┐ ┌────────────────────────┐ │   │
│  │  │ Receipt      │ │ Data Table  │ │ ChatSidebar            │ │   │
│  │  │ Uploader     │ │ (4 views)   │ │ (SSE + tool approval)  │ │   │
│  │  └──────┬───────┘ └──────┬──────┘ └────────────┬───────────┘ │   │
│  └─────────┼────────────────┼─────────────────────┼─────────────┘   │
│            │                │                     │                  │
└────────────┼────────────────┼─────────────────────┼──────────────────┘
             │                │                     │
      ┌──────┘       ┌───────┘           ┌──────────┘
      ▼              ▼                   ▼
┌───────────┐  ┌───────────────┐  ┌──────────────┐  ┌───────────────┐
│ Storage   │  │ Data Table    │  │ OpenRouter   │  │ Google Cloud  │
│ Brain SDK │  │ React +       │  │ (LLM API)   │  │ Vision API    │
│           │  │ Prisma Adapter│  │              │  │ (OCR)         │
└─────┬─────┘  └───────┬───────┘  └──────────────┘  └───────────────┘
      │                │
      ▼                ▼
┌───────────┐  ┌───────────────┐
│ Object    │  │ PostgreSQL    │
│ storage   │  │               │
└───────────┘  └───────────────┘

Page Structure

Upload Page (`/`)

Drag-and-drop zone for images and PDFs (multi-file selection supported)
Batch upload queue: files are processed sequentially through upload, OCR, classify, and save phases
Per-file progress indicators with phase-level detail
Failed files do not block the remaining queue
Automatic redirect to dashboard when all files complete

Dashboard (`/dashboard`)

Powered by @marlinjai/data-table-react
4 pre-configured views:
- Table -- grouped by Category (default)
- By Konto -- grouped by SKR03 account number
- Board -- Kanban-style board grouped by Status
- Calendar -- date-based view using the Date column
Column management, multi-row selection, search, filter, pagination
Inline cell editing
AI Chat sidebar toggle

Data Flow

Upload Flow (Batch)

Users can select multiple files at once. Each file is added to a queue and processed sequentially through the full pipeline. Failed files do not block subsequent files.

User drops one or more images/PDFs (or clicks to browse)
      │
      ▼
Files added to upload queue (QueueItem[])
      │
      ▼
┌─── For each file in queue (sequential) ───────────────────────┐
│                                                                │
│  Phase 1: Upload to Storage Brain (R2)                         │
│        │                                                       │
│        ▼                                                       │
│  Phase 2: POST /api/ocr with fileId                            │
│        │   Fetch file from Storage Brain →                     │
│        │   send to Google Cloud Vision API                     │
│        │   (images: images:annotate, PDFs: files:annotate)     │
│        ▼                                                       │
│  Return OcrResult { fullText, blocks, confidence }             │
│        │                                                       │
│        ▼                                                       │
│  extractReceiptFields(ocrResult) — heuristic extraction        │
│        │   → vendor, gross, net, taxRate, date, category,      │
│        │     konto, name                                       │
│        ▼                                                       │
│  Phase 3: POST /api/classify-single (AI classification)        │
│        │   → category, konto, zuordnung, confidence, reasoning │
│        ▼                                                       │
│  Phase 4: Create row in receipts table via Prisma adapter      │
│        │                                                       │
│        ▼                                                       │
│  File marked done (or error) — next file begins                │
└────────────────────────────────────────────────────────────────┘
      │
      ▼
All files processed → redirect to dashboard

AI Chat Flow

User opens chat sidebar → types message
      │
      ▼
POST /api/chat (streaming SSE)
      │   system prompt includes: table schema, select options,
      │   SKR03 mappings, zuordnung options, user rules
      ▼
LLM responds with text + optional tool_calls
      │
      ▼
Frontend receives SSE events:
      ├── text_delta → rendered incrementally
      ├── tool_use  → displayed as pending action
      │       │
      │       ├── read-only tool (get_rows, get_columns, get_select_options)
      │       │       → auto-executed, result sent back as tool_result
      │       │
      │       └── write tool (update_cells, bulk_update, create_row, delete_rows)
      │               → requires user approval ("Apply" / "Apply All")
      │               → on approval: executed client-side, result sent back
      │
      └── done → response complete

Field Extraction Engine

Located at src/lib/extract-receipt-fields.ts (~500 lines). Returns an ExtractionResult with name, vendor, gross, net, taxRate, date, category, and konto.

Amount Extraction (multi-pass)

Net: looks for lines matching subtotal/netto/before-tax labels
Tax: looks for tax/VAT/MwSt labels (excluding total lines)
Gross (4 passes):
- High-priority: "grand total", "amount due", "balance due"
- Medium-priority: generic "total" (excluding subtotal/tax lines)
- EU keywords: "gesamt", "summe", "brutto"
- Fallback: largest amount found anywhere in the text
Derivation: if 2 of 3 values are found, the third is calculated

Vendor Extraction

Primary: spatial extraction from OCR bounding boxes (topmost non-noise block)
Fallback: first non-noise line in the first 8 lines of OCR text
Noise filter: skips pure numbers, addresses, metadata labels, generic headings

Date Extraction

Priority: labeled dates ("Date:", "Invoice Date:") first
Formats: ISO (2024-01-15), EU dot (15.01.2024), US slash (01/15/2024), named months (Jan 15, 2024)
Skips expiry/card dates

Category Inference (3-pass)

Vendor lookup: matches vendor name against ~80 known vendors (e.g., "starbucks" -> Bewirtung)
Keyword scan: matches full OCR text against category keyword patterns
Item patterns: checks for specific line-item hints (e.g., "cappuccino" -> Bewirtung)
Falls back to "Sonstige Ausgaben" if no match

Prisma Adapter

The app uses @marlinjai/data-table-adapter-prisma to persist the dynamic dt_* tables in PostgreSQL. The adapter is constructed with the shared Prisma client:

// src/app/app/layout.tsx
import { PrismaAdapter } from '@marlinjai/data-table-adapter-prisma';
import { prisma } from '@/lib/prisma';

const adapter = new PrismaAdapter({ prisma });

The connection comes from DATABASE_URL, and the schema plus its migrations live in prisma/schema.prisma and prisma/migrations/.

Receipt Table Schema

Column	Type	Description
`Name`	text	Composite summary (primary column)
`Vendor`	text	Merchant name (OCR spatial extraction)
`Gross`	number	Total amount incl. tax
`Net`	number	Amount before tax
`Tax Rate`	number	Tax percentage (e.g. 19 for 19%)
`Date`	date	Receipt date (ISO 8601)
`Category`	select	SKR03 expense category (10 options)
`Konto`	text	SKR03 account number (e.g. "4650")
`Status`	select	Pending / Processed / Rejected
`Confidence`	number	OCR or AI classification confidence
`Receipt Image`	url	Link to original file in Storage Brain
`OCR Text`	text	Raw OCR text for AI context
`Zuordnung`	select	Dynamic column: Universitat / Geschaftlich / Privat

SKR03 Category-to-Konto Mapping

Category	Konto
Bewirtung	4650
Reisekosten	4670
Burobedarf	4930
Software & Lizenzen	4806
Telefon & Internet	4920
Hardware & IT	4855
Miete & Nebenkosten	4210
Versicherungen	4360
Fachliteratur	4940
Sonstige Ausgaben	4900

API Routes

Route	Method	Purpose
`/api/ocr`	POST	Fetches file from Storage Brain, sends to Google Cloud Vision, returns `OcrResult`
`/api/classify-single`	POST	LLM classification of a single receipt via OpenRouter
`/api/chat`	POST	Streaming AI chat with tool use (SSE)
`/api/files/[fileId]`	GET	Proxies file downloads from Storage Brain

Environment Configuration

# Storage Brain (file uploads to R2)
NEXT_PUBLIC_STORAGE_BRAIN_API_KEY=sk_live_...
NEXT_PUBLIC_STORAGE_BRAIN_URL=https://storage-brain-api.marlin-pohl.workers.dev

# Google Cloud Vision (OCR)
GOOGLE_CLOUD_VISION_API_KEY=AIza...

# OpenRouter (AI classification + chat)
OPENROUTER_API_KEY=sk-or-v1-...

# Optional: override AI models
# AI_MODEL=anthropic/claude-sonnet-4.6            # chat sidebar
# AI_CLASSIFY_MODEL=anthropic/claude-haiku-5.5    # reads a receipt, page and text; must accept image and file input

Database connectivity comes from the DATABASE_URL environment variable, injected at runtime by the Infisical CLI (see Dockerfile).

Deployment

Target: Docker image (see Dockerfile)

The app is deployed at receipts.lumitra.co. Server-side secrets (DATABASE_URL, GOOGLE_CLOUD_VISION_API_KEY, OPENROUTER_API_KEY, the OpenFGA credentials) live in Infisical and are injected into the container at start, never baked into the image. Client-side env vars use the NEXT_PUBLIC_ prefix.