fshy / Crawling

Internet Archive Wayback Machine

Crawling · 1 record · JSON

Internet Archive Wayback Machine Crawling

★ qualifiesanonymousno cardMper-object-cap

web.archive.org/ · archive · crawler-scraper/wayback · JSON · compare

Free tier	keyless listing, closest and replay of existing captures; Save Page Now needs keys unit requests; metered on request-count
Price	none stated
Per-object cap	not applicable: the unit is a capture
Retention	captures are held indefinitely; a listing row's record_length is the compressed WARC size, not the body
Egress	served through the broker; a direct replay of a large object is not measured here
Identity	anonymous listing, closest and replay need nothing; Save Page Now needs the operator's Internet Archive keys
Limits	request-rate Save Page Now is held after 5 saves of one URL a day on this deployment; the listing and replay are keyless per-object-cap a listing pages by a CDX resume key; a page that holds limit captures names a cursor for the next
Binding	per-object-cap
Interface	rest CDX server, Availability API and raw replay with an id_ suffix
Tooling	not stated
Composes with	Rung four of the escalation ladder; the copy that survives the origin changing, as hostloc.com did
Notes	The single most useful service in this topic and the only one that answered within 4 s on every call. A short at= is the start of its period, so at=2024-06 asks 20240601000000.
Evidence rung	measured
Checked	2026-10-22 (live); stale on 2026-12-21

Evidence (3)

date	source	what it proved
2026-10-22	web.archive.org/web/202406/hostloc.com/forum.php	200 in 3.2 s; capture 20240530023233, status 200, mimetype text/html, record_length 7402, compared[] names a distance of 163647 s
2026-10-22	web.archive.org/web/20240530023233/https://hostloc.com/forum	200, 26,000 bytes, x-web-origin-status 200, x-web-body archived: a 2024 copy of a forum that no longer serves its forums
2026-10-22	web.archive.org/web/*/example.com	3 captures then 3 more by next, no timestamp repeated, more true on both pages

Unknown or withheld values (4)

$.limits[0].source private-url
$.limits[1].source private-url
$.reliability_note withheld
$.tooling not-stated

Built from commit 872967ae41, data as of 2026-10-27. Every value links to its source; freshness badges are computed in your browser. How to read this.