Markdown
# 4chan & Imageboard Archiving Resources

## Thread Archivers

| Project | Language | Purpose | Notes |
|---------|----------|---------|------|
| archive-chan | Python | Archive individual threads | Maintained fork of BASC-Archiver; downloads HTML, JSON, images, thumbnails. Does **not** preserve deleted posts across updates. :contentReference[oaicite:0]{index=0}
| BASC-Archiver | Python | Original thread archiver | Historical predecessor to archive-chan. :contentReference[oaicite:1]{index=1}
| 4TCT | Python | Research-oriented collector | Stores text from selected boards via the 4chan API; intended for academic data collection rather than full archival. :contentReference[oaicite:2]{index=2}

---

# Board Archivers

These archive entire boards continuously rather than individual threads.

| Project | Language | Purpose |
|---------|----------|---------|
| Fuuka | Perl | Original 4chan board archiver |
| Asagi | Java | High-performance reimplementation of Fuuka; backend scraper used by many archives |
| FoolFuuka | PHP | Modern frontend for Asagi/Fuuka archives with search, ghost posts, etc. |

These are the software stacks that preserve deleted posts **if they were scraped before deletion**. :contentReference[oaicite:3]{index=3}

---

# Public Archive Sites

These sites are useful references for studying archive implementations.

| Site | Software | Notes |
|------|----------|------|
| archived.moe | FoolFuuka + Asagi | Multi-board archive; "last resort" archive for many boards. :contentReference[oaicite:4]{index=4} |
| Desuarchive | FoolFuuka | Long-running archive with annual database exports. :contentReference[oaicite:5]{index=5} |
| AyaseQuart | Asagi scraper + custom frontend | Archives many currently active boards. :contentReference[oaicite:6]{index=6} |
| Nyafuu (historical) | Fuuka | Shut down in 2022; data transferred to successor archives. :contentReference[oaicite:7]{index=7} |
| Foolz.us (historical) | FoolFuuka | Influential early archive; migrated to archive.moe. :contentReference[oaicite:8]{index=8} |
| archive.moe (historical) | FoolFuuka | Successor to Foolz.us. :contentReference[oaicite:9]{index=9} |
| 4ch.be / FullFuuka (historical) | FullFuuka | Archived all boards with thumbnails and published weekly dumps. :contentReference[oaicite:10]{index=10} |

---

# Datasets

## Large Public Datasets

- Raiders of the Lost Kek (3.3 million threads / 134 million posts)
- Annual Desuarchive database dumps
- Historical FullFuuka weekly dumps
- Various Internet Archive mirrors of retired archives

:contentReference[oaicite:11]{index=11}

---

# Articles & Documentation

## ArchiveTeam Wiki

Probably the single best historical overview.

Topics include:

- history of Fuuka
- Asagi development
- FoolFuuka
- archive genealogy
- discontinued archives
- archival philosophy
- surviving database dumps

:contentReference[oaicite:12]{index=12}

---

## Research Papers

### 4TCT: A 4chan Text Collection Tool (2023)

Describes a Python collector using the official API for research.

:contentReference[oaicite:13]{index=13}

---

### Raiders of the Lost Kek (2020)

Large released dataset of archived /pol/ threads including deleted content.

:contentReference[oaicite:14]{index=14}

---

### Understanding Web Archiving Services and Their (Mis)Use on Social Media (2018)

Discusses archive.is, Wayback Machine, Reddit, Twitter, Gab and 4chan usage patterns.

:contentReference[oaicite:15]{index=15}

---

# Community Discussions

## DataHoarder

Useful discussions on:

- archive recommendations
- historical datasets
- Desuarchive exports
- missing early archives

:contentReference[oaicite:16]{index=16}

---

# Interesting Historical Notes

- Fuuka was first written in 2007.
- Asagi is a Java rewrite focused on scalability.
- FoolFuuka became the dominant frontend used by public archives.
- Modern public archives generally run **Asagi + FoolFuuka**.
- Most thread archivers (archive-chan, BASC-Archiver) are **snapshot-based**, whereas board archivers are **incremental**.

:contentReference[oaicite:17]{index=17}

---

# If You Want an Incremental Thread Archiver

There does **not** appear to be a well-maintained standalone tool that:

- polls a single thread,
- merges new posts,
- preserves deleted posts,
- marks posts as deleted instead of removing them,
- regenerates HTML from the merged state.

Current options are:

1. Run Asagi (heavyweight).
2. Write a custom poller using the JSON API.
3. Patch archive-chan with an append-only merge strategy (likely the simplest path).