In August 2026, the Macedonian URL shortener 0.mk team did something rare: they pulled from the database all the links created between 2009 and 2014 and tried to open each one. There were 657,607 links; 655,178 of them were allowed to be crawled (2,429 records were discarded as broken or blocked by the service rules). The crawler went up to five redirects and retried from the second network if the connection failed; a response of 2xx or 3xx was considered a success.
Open rate: 23.32%.
The remaining three quarters are no longer pages, but traces. And the most interesting thing here is not the share itself, but its structure: more than half of the links (51.24%) died at the network level — the hostname could not be resolved, the server did not respond, TLS negotiation failed. In other words, the document did not disappear, the entire host did. Another 25.44% responded with HTTP errors, of which 76 are 403 — honest 404s. Of 133,605 unique hosts, at least one live URL was found only on 34,827. For 98,778 domains, nothing works.
Before you take these percentages for yourself: this is a sample from a single URL shortener in one country over five specific years. The authors explicitly state that they describe the Macedonian segment, not the global web. The year 2011 with its 92.53% dead links is skewed by a single party with 83,398 links on one domain. The order of magnitude of this caveat does not cancel out comparable results from other LinkRank studies, but the exact figures are not universal.
The authors’ overall conclusion sounds grim for anyone who loves small sites: the centralized web has survived about a decade and a half much better than the small one. YouTube and Wikipedia remain in place. Personal blogs, forums, photo hosts — not so much.
For OSINT this is not philosophy, but a practical problem. A report in which the source is given by a link becomes, after a year, a set of statements without verification. And it breaks not due to your fault and without warning: the site owner runs out of hosting, the domain is resold, the engine moved and the URL scheme changed. Below are five levels of what can be done about this, from zero to excessive. The meaning is in the levels themselves: you don’t need to build an archival cluster right away; you need to understand which step you are on and what the next one gives you.
Level 0. Link in the note
This is where most of them are. The link is copied into a document, into Notion, into a Telegram channel, at the end of the report. There is no copy of the content.
The link is not a document, but an address that the document was promised to be kept at. The promise is one-sided and free for the one who gave it. That is exactly why the share of surviving links over 15 years in the 0.mk study is 23%.
A separate trap of Level 0 is a screenshot. A screenshot looks like a copy, but it is not: it has no URL in machine-readable form, no timestamp, no HTTP headers, no source HTML. As an illustration it’s fine. As a source citation — not.
Level 1. Someone else’s archive
The cheapest next step: ask someone who does this professionally to preserve the page. Practically, Wayback Machine and archive.today.
Wayback Machine has a Save Page Now form at archive.org/web/: paste the address, you get a permanent link to the snapshot. Important limitations are described in the project’s help:
- only one page is saved at a time, not the whole site;
- images and CSS of that page are saved, outgoing links are not;
- the visitor’s IP is not stored, the submission is anonymous;
- some sites block crawling, TLS settings may break.
Practical tip: save not only the target page but also the pages you plan to link to within it. Otherwise in a year you’ll have a live article snapshot with dead links throughout the text.
The main drawback of Level 1 is you don’t control the storage. A snapshot can be deleted at the copyright holder’s request, a page can be excluded from the archive retroactively, and the archive itself may become unavailable in your country. A third-party archive is great as a duplicate and bad as the sole copy.
Therefore Level 1 makes sense not instead of the others, but in addition: it’s public and you can cite it in the text, something you can’t do with a file on your own disk.
Level 2. A single self-contained file
The first step where the copy actually sits with you.
monolith — a small utility in Rust that assembles a page into a single HTML file: CSS, images, and scripts are embedded as data-URL. You get a document that opens offline and looks the same as the original.
# Debian/Ubuntu — from cargo, repository version is usually old
cargo install monolith
# save the page in full
monolith https://example.com/article -o article.html
# without scripts and fonts — smaller file and safer to open
monolith -j -F https://example.com/article -o article.html
# placeholders in the output filename
monolith https://example.com/article -o '%title%.%timestamp%.html'
Useful flags: -j — drop JavaScript, -c — no CSS, -i — no images, -F — no web fonts, -v — no video, -a — no audio, -I — isolate document (block any external requests), -d/-B — white and blacklists of resource domains, -m — save in MHTML, -u — your User-Agent, -t — timeout. The current version at the time of writing is 2.10.1 as of March 30, 2025.
The key limitation of monolith is stated in its own README: it doesn’t have its own JavaScript engine. It grabs the HTML as served by the server. Everything rendered on the client — endless feed, comments, content of a single-page app — will not be captured in the file. Saving a social feed or an SPA dashboard this way and assuming the copy is complete is a common mistake.
Where monolith struggles, the SingleFile extension for Firefox and Chromium comes in with the same idea but runs inside the browser. It saves the already-rendered DOM, with everything that JavaScript loaded, and in your session — i.e., a page behind a login as well. The catch: this is a manual browser action; automating it is more challenging.
Level 3. Your own archival service
When you need to save regularly and in large volume, manual utility runs stop working. This is where ArchiveBox appears — a self-hosted archiver that, for every URL, runs several extractors and stores the results in a folder structure plus SQLite.
What it saves for a single address: a SingleFile snapshot, a PDF, a PNG screenshot, raw DOM, WARC via wget, text extraction (htmltotext), readable version (readability, mercury), favicon, media via yt-dlp, a clone of the repository for links to GitHub/GitLab — and, if allowed, it duplicates a copy to archive.org.
mkdir -p ~/archivebox/data && cd ~/archivebox
curl -fsSL 'https://docker-compose.archivebox.io' > docker-compose.yml
docker compose run archivebox init --install
docker compose up
```The web interface will run on port 8000. Further URLs can be submitted not only by hand: history and browser bookmarks import, exports from Pocket and Pinboard, RSS feeds, a browser extension, and scheduled runs are supported.
[info]
**Requirements you should know in advance.** ArchiveBox pulls in Chrome (for screenshots and PDFs), `wget`, `yt-dlp`, Python ≥ 3.13, Node ≥ 22, `curl`, `git`, and `ripgrep`. Dependencies are modular — you can skip unnecessary extractors. Architectures: amd64, arm64 and arm7, which means Raspberry Pi 3 and newer also work, although rendering in Chrome on it will be slow.
[/info]
The main resource here is the disk, and you should plan for it based on which extractors are enabled. A full set for a single article is a single-file copy, PDF, full-page screenshot, and WARC; together, this easily amounts to several megabytes per URL, and if the article has embedded video and the media extractor is enabled, it can run into hundreds of megabytes. An estimate of “one hundred links per week” becomes tens of gigabytes per year. Disabling unnecessary extractors is cheaper than cleaning up later.
## Level 4. WARC and evidentiary value
Levels 2 and 3 give you a copy that is easy to read. Level 4 — a copy that you can present as evidence.
WARC (Web ARChive) — the Library of Congress format and de facto standard for web archiving: it stores not a rendered page, but the HTTP transactions themselves — requests, responses, headers, bodies, timestamps. That is why WARC is more valuable than an HTML file: it shows what the server actually replied and when.
The simplest way to obtain a WARC is the usual `wget`:
```bash
wget --warc-file=snapshot-2026-08-14 \\
--warc-cdx \\
--page-requisites \\
--no-warc-compression \\
--user-agent="Mozilla/5.0 (research archiving)" \\
https://example.com/article
For sites as a whole and pages with heavy JavaScript there is Browsertrix from Webrecorder — a crawler that drives a real browser and writes the result to WARC/WACZ; this is already a full service, not a utility.
Next comes the part that turns a file into evidence:
# archive fingerprint
sha256sum snapshot-2026-08-14.warc > snapshot-2026-08-14.warc.sha256
# and a record of the facts: what, where, when, with what
cat > snapshot-2026-08-14.txt <<'EOF'
URL: https://example.com/article
Date (UTC): 2026-08-14T05:40:12Z
Tool: wget 1.24.5, --warc-file
Operator: <who captured>
SHA-256: <from .sha256>
EOF
The hash calculated by you and placed next to the file proves only integrity — that the file has not changed since the hash was computed. It does not prove that the archive was captured on the stated date: you entered the date yourself. If the capture needs to stand up in a dispute or a legally binding report, the fingerprint should be anchored with an independent third party — a trusted timestamping service (RFC 3161), notarized, or at least posted to a public channel with external dating. And separately: digital evidence requirements vary by jurisdiction; here we describe the technical minimum, not the legal procedure.
Decision Matrix
| What it protects | Lives without you | JS and login | Costs | Suitable as evidence | |
|---|---|---|---|---|---|
| 0. Link | nothing | — | — | 0 | no |
| 1. Wayback / archive.today | the fact of publication | yes, but not by you | partial | 0 | weak, but publicly verifiable |
| 2. monolith | text and layout | no | no | seconds | no |
| 2. SingleFile | rendered page | no | yes | manual click | no |
| 3. ArchiveBox | everything at once, in bulk | no | partial | server + disk | partial (WARC inside) |
| 4. WARC + timestamp | HTTP transactions | no | depends on crawler | noticeable | yes |
A normal working configuration for OSINT tasks is not one row from the table, but two: level 1 for a public link in the report plus level 2 or 3 for your own copy. Level 4 is reached when the material may be needed in a dispute.
Where this all breaks
Dynamics and login. Everything behind authentication is saved only from the browser with your session — that is, SingleFile or Browsertrix with a profile. And here the question arises what exactly you captured: if your cookies or personal data of third parties ended up in the snapshot, you must store such an archive accordingly.
Site versus archive. Some resources are intentionally blocked by archivers — by User-Agent, by robots.txt, by CAPTCHA, by behavioral signals. The flip side of the same coin was discussed in the topic Eight lines of defense a site has against AI scrapers: site owners do not distinguish your archiver from a scraper training another model, and defend against all of them at once.
Volume. An archive without rotation and without a budget for disk ends up being deleted entirely. Decide in advance what to keep forever and what to keep for a year.
Verification. The most common disappointment is discovering a year later that the “saved” page does not open: the file is corrupted, the extractor fell silently, a captcha ended up in the snapshot instead of content. Opening a copy with the network disabled is the only way to ensure it is self-contained.
Minimal discipline that pays off
- Save the source at the moment you found it, not when you sit down to write the report. Backdating a snapshot is not possible: the page is already different.
- Put a nearby text file with: URL, date and time in UTC, tool, who captured it.
- Archive Wayback everything that is publicly accessible — this is free insurance and a public link for the text.
- Once per quarter, open several random offline copies from the archive and verify that they are alive.
- Keep the archive where the rest of your important data sits, and under the same rules: three copies, two media, one off-site.
Sources
- 0.mk — Where did the old web go? We followed 657,607 links to find out — methodology, year-by-year and type-of-failure breakdown
- monolith on GitHub — list of keys, installation methods, note on absence of JS engine
- ArchiveBox on GitHub — extractors list, docker-compose quickstart, dependencies and supported architectures
- Save Page Now — Internet Archive help — what exactly is saved and what is not
- SingleFile on GitHub — extension for saving a rendered page
What level are you at now — and what brought you here? Also: who has ArchiveBox run the longest for over a year, and how much disk space did it end up consuming on a real data stream of links?