Skip to content

Files

ARCHITECTURE.md

Latest commit

rithwikgokhalecursoragent
rithwikgokhale
and
cursoragent
Add IMPLEMENTATION.md handbook for Phase 2-4 work
May 9, 2026
0f48d98 · May 9, 2026

History

History
136 lines (102 loc) · 8.21 KB

ARCHITECTURE.md

File metadata and controls

136 lines (102 loc) · 8.21 KB
Raw

Architecture

Source of truth for how the v1 extension is structured. Update this file whenever a phase materially changes the design.

For step-by-step implementation guidance for Phases 2-4, see IMPLEMENTATION.md.

Goals

  1. $0 ops cost. No backend the maintainer has to run, monitor, or pay for.
  2. Privacy by default. No data leaves the user's machine in the local tier.
  3. Honest accuracy story. When local model confidence is low or the user enables Boost mode, the UI says so explicitly. No false certainty.
  4. Click-to-scan UX. Nothing happens automatically — the user always pulls the trigger.
  5. Web Store reviewable. Minimum viable permissions, clear single-purpose, public privacy policy.

High-level flow

 
flowchart LR
  Popup[popup React app] -->|scanImages / scanText| BG[background<br/>service worker]
  BG -->|inject| CS[content script]
  CS -->|find visible imgs / selection| CS
  CS -->|fetch image bytes via fetch + canvas| CS
  CS -->|classify request| BG
  BG -->|relay| OFF[offscreen document]
  OFF -->|ONNX inference| Local["transformers.js<br/>local model"]
  OFF -.HTTPS, opt-in.-> Provider["Gemini / OpenAI / HF<br/>BYO API key"]
  Local -.cached.-> IDB[(IndexedDB)]
  Local -.first run.-> HFCDN[HF CDN] -.fallback.-> GHRel[GitHub Releases]
  OFF -->|result| BG -->|badge update| CS

Components

Component Lives in Job
Popup src/popup/ Two big buttons: Scan visible images, Scan selected text. Recent-results list with thumbnails. Indicator showing whether local model is downloaded + which provider is active.
Options page src/options/ Provider selector + API key input + test button. Confidence threshold slider. Local-model management (re-download / clear cache). Telemetry opt-in.
Content script src/content/ (a) Discovers visible images / text selection. (b) Fetches image bytes via fetch(src, { credentials: 'include' }) → createImageBitmap → 224×224 canvas → ImageData. (c) Renders badges + highlights as a small React island. (d) Reuses the v0.1 message-passing pattern with requestId correlation, ResizeObserver, MutationObserver.
Background SW src/background/ Routes typed messages (ScanImagesRequest, ClassifyImage, Result, …). Owns the offscreen-document lifecycle (creates it on first scan, tears it down on idle). Routes per-image classify calls to the offscreen doc.
Offscreen document src/offscreen/ Holds the long-lived ONNX session(s). Service workers can't reliably hold large WASM/WebGPU contexts; offscreen documents can. Lazy-inits transformers.js pipelines. Handles BYO-provider HTTP calls too (so background SW stays minimal).
Detector router src/detectors/ Provider abstraction: local, gemini, openai, hf. Each implements classify(input): Promise<Result>. Router checks user config and falls back gracefully.
Storage layer src/shared/storage.ts Typed wrapper over chrome.storage.sync (settings) and chrome.storage.local (recent results, model state).
Result cache src/shared/cache.ts IndexedDB. Keyed by SHA-256 of image bytes (not URL — same image at different URLs counts once). 7-day TTL.

Why an offscreen document

Manifest V3 service workers have aggressive idle timeouts (30s) and limited APIs. transformers.js needs to:

  • hold a multi-hundred-MB WASM/WebGPU context across calls,
  • persist between idle moments (the user might scan once, scroll, then scan again 5 min later),
  • access OffscreenCanvas / WebGPU / IndexedDB reliably.

Offscreen documents solve all three. The background SW creates one on first scan and reuses it.

Message bus

All inter-context messages are typed via a discriminated union in src/shared/types.ts:

type Message =
  | { type: 'SCAN_IMAGES'; tabId: number }
  | { type: 'SCAN_TEXT'; text: string; tabId: number }
  | { type: 'CLASSIFY_IMAGE'; requestId: string; bytes: ArrayBuffer; mime: string }
  | { type: 'CLASSIFY_TEXT'; requestId: string; text: string }
  | { type: 'RESULT'; requestId: string; result: ClassificationResult }
  | { type: 'MODEL_STATUS'; phase: 'idle' | 'downloading' | 'ready' | 'error'; pct?: number };

requestId correlates badge updates back to the right image/paragraph in the content script — same pattern as v0.1.

Permissions (minimum viable)

{
  "permissions": ["activeTab", "scripting", "storage", "offscreen", "contextMenus"],
  "host_permissions": []  // none — activeTab is granted on click
}

Notably no <all_urls>. We don't need it — activeTab is granted on the user's click, which matches the click-to-scan UX exactly. Faster Chrome Web Store review, less scary install dialog.

Model distribution

  • Primary: Hugging Face hub via transformers.js (env.remoteHost = 'https://huggingface.co'). Model pinned to a specific revision SHA so behavior is deterministic.
  • Fallback: GitHub Releases on rithwikgokhale/image-ai-detector. Each release attaches the ONNX weights as release assets via scripts/mirror-model.sh. transformers.js's env.remoteHost is overridden on a fetch failure.
  • Cache: transformers.js stores the model in IndexedDB by default. We expose a "Clear cached model" button in Options for debugging.
  • Quantization: all models ship at q8 (int8) — ~4× smaller than fp32, near-lossless accuracy.
  • Backend: device: 'auto' — WebGPU when available, WASM fallback.

Default models (locked-in)

Modality Model Size at q8 Decision rationale
Image Organika/sdxl-detector ~22 MB Most popular open AI-image detector with strong transformers.js support. Phase 2.5 benchmark may swap.
Text Hello-SimpleAI/chatgpt-detector-roberta ~32 MB Most-used open text detector. Known limitations on Claude/Gemini text — bias toward "Uncertain" with high default threshold.

Power users can override either via localImageModel / localTextModel in chrome.storage.sync.

BYO API key (Boost mode)

Each provider has a tiny adapter under src/detectors/{image,text}/{provider}.ts. They all return the same ClassificationResult shape so the UI is provider-agnostic.

Provider Image Text Notes
Google Gemini 2.5 Flash ✅ ✅ Free tier ≈1500 req/day. Headline integration.
OpenAI GPT-4o-mini GPT-5-mini Cheap, accurate, structured outputs.
Hugging Face Inference API ✅ (any classifier) ✅ (any classifier) BYO HF token. Lets advanced users plug in newer detection models without waiting for an extension release.

API keys live in chrome.storage.sync (encrypted at rest by Chrome, synced if the user has Chrome sync enabled). They are sent only to the chosen provider — never to any server we operate. There is no "we" server.

Failure behavior (locked): if a Boost call fails (rate limit, bad key, network), the router falls back to the local model and surfaces a visible toast: "Boost failed (rate limited) — using local model". Toggleable in Options (default on).

Rate limiting (locked): client-side, 10 RPM per provider by default, queued and trickled. Configurable in Options.

Result shape

type ClassificationResult = {
  label: 'ai' | 'real' | 'uncertain';
  confidence: number;          // 0..1
  source: 'local' | 'gemini' | 'openai' | 'hf';
  modelId: string;             // e.g., "Organika/sdxl-detector@a1b2c3"
  rawScores?: Record<string, number>;
  reasoning?: string;          // BYO providers can return this; local does not
  durationMs: number;
};

label === 'uncertain' when confidence falls in the user's "uncertain band" (default 0.45–0.55). The badge shows a yellow "Uncertain" pill instead of forcing a wrong call.

What this is not doing (and why)

  • No automatic scanning as you scroll. Click-to-scan only. Privacy + perf + UX.
  • No hosted backend. $0 ops, no abuse vector, no privacy policy headaches beyond the providers users explicitly opt into.
  • No accounts, no auth, no payments in v1. Possibly later if BYO is too friction-y for non-technical friends.
  • No <all_urls> host permission. Activetab + scripting suffice for click-to-scan.