Software Heritage

softwareheritage.org · Storage · 2 records · JSON

Software Heritage Archive (CAPI read side) + Save Code Now (anonymous crawl trigger) Storagecaveatinstitutionno cardRper-object-cap
software heritage deposit (swh-deposit, the push api) Storagefails: quotaemail?card?R

Software Heritage Archive (CAPI read side) + Save Code Now (anonymous crawl trigger) Storage

caveatinstitutionno cardRper-object-cap

archive.softwareheritage.org · academic · storage/software-heritage-archive-save-code-now · JSON · compare

Free tier	No storage quota is imposed on the depositor, because the depositor never uploads bytes. The archive stores content-addressed git objects from public repositories, deduplicated by sha1_git, and growth is the Foundation's problem, not yours. Live scale reference: the `torvalds/linux` origin is at visit 500 ('visit':500, date 2026-10-18). Deposit (push) is separate and capped: swh-deposit docs say 'upload limitation of 100Mib', and the source default in swh/deposit/config.py is "max_upload_size": 209715200 (200 MiB). unit mixed; metered on stored-bytes
Price	none stated
Per-object cap	per-object-cap (deposit only): 100 MiB (104857600 bytes)
Retention	list-rate-limited, and this is the binding constraint on reading a large tree (walking a repo = 1 call per directory). retention: permanent and intrinsic - this is a preservation archive, deletion is not part of the model; `visit_status: full` snapshots are retained per visit
Egress	Free and unmetered - no pricing page, no billing, a public research archive. Downloads are direct `GET .../raw/` byte streams from Varnish. Note there is no CDN requirement to satisfy: nothing to pay.
Identity	institution derived from the record's prose, not from an identity field, and no signup was executed: content addressable and permanent. The mechanism: POST /api/1/origin/save/git/url/<URL>/ anonymously, with no account and no key, and it returns {"save_task_status":"pending","loa
Limits	per-object-cap per-object-cap (deposit only): 100 MiB source per-object-cap (deposit only): 100 MiB documented, 200 MiB default in swh-deposit config. request-rate: SWH CAPI sends `X-Ratelimit-Limit: 120` / `X-Ratelimit-Remaining` / `X-Ratelimit-Reset` (observed 114->118 remaining across calls) - 120 requests per window per IP, i.e. list-rate-limited, and this is the binding constraint on reading a large tree (walking a repo = 1 call per directory). retention: permanent and intrinsic - this is a preservation archive, deletion is not part of the model; `visit_status: full` snapshots are retained per visit. Write access: /api/1/origin/save/ is open anonymously; deposit.softwareheritage.org returns 401 to anonymous PUT/POST.
Binding	per-object-cap the per-object cap sets the shard size and therefore the object count; it bounds the total more tightly than any headline quota
Interface	rest, git, rclone rest
Tooling	Official Web API v1 (documented route index at https://archive.softwareheritage.org/api/1/); Python `swh.model`/`swh.web.client`; no rclone backend. Read paths: /api/1/origin/<url>/get/, /api/1/origin/<url>/visit/latest/, /api/1/snapshot/<id>/, /api/1/revision/<id>/, /api/1/directory/<id>/, /api/1/content/sha1_git:<h>/raw/.
Composes with	This is the strongest chaining object in the niche and it is NOT a place to upload - it is a place to make arbitrary public git content addressable and permanent. The mechanism: POST /api/1/origin/save/git/url/<URL>/ anonymously, with no account and no key, and it returns {"save_task_status":"pending","loading_task_id":422136312}. So: unlimited free git hosts (one repo each, any size) -> anonymous SWH crawl request per URL -> content permanently in SWH under a sha1_git you can fetch forever. The upload-to-SWH step costs zero bytes of your quota; the bytes live in the free git host and SWH becomes the durability + retrieval layer. Multiply by unlimited origins. Combine with round-1's GitHub Releases finding: the archive is a free permanence layer over storage you already have.
Reliability	Read path is reliable but rate-shaped: 120 req/window/IP will throttle a naive tree walk of a 20 TiB corpus - parallelise across IPs or batch directory listings. docs.softwareheritage.org sits behind Anubis PoW (proof-of-work bot wall) and returns a challenge page with upstream_status 200, so doc numbers must be taken from the GitHub source, not the docs site. Anonymous save requests are queued ('pending'), not instant - ingestion lag is hours to days.
Notes	No storage quota is imposed on the depositor, because the depositor never uploads bytes. The archive stores content-addressed git objects from public repositories, deduplicated by sha1_git, and growth is the Foundation's problem, not yours. Live scale reference: the `torvalds/linux` origin is at visit 500 ('visit':500, date 2026-10-18). Deposit (push) is separate and capped: swh-deposit docs say 'upload limitation of 100Mib', and the source default in swh/deposit/config.py is "max_upload_size": 209715200 (200 MiB).
Evidence rung	reported One page read moves it to documented.
Checked	2026-10-21 (inherited); stale on 2026-11-04
Parsed numbers	`$.limits[0].value` limit:per-object-cap 100.0 MiB

Evidence (6)

date	source	what it proved
2026-10-21	archive.softwareheritage.org/api/1/origin/https://github.com	probe: GET -> 200, {"visit":500,"date":"2026-10-18...","snapshot":"221b3be5a85f08858a1410e5987f88e77603adb1","status":"full","type":"git"}
2026-10-21	withheld	probe: GET /api/1/snapshot/221b3be5.../ -> 200, branches include HEAD + refs/heads/master + refs/pull/10/head (full graph, anonymous)
2026-10-21	withheld	probe: GET /api/1/content/sha1_git:010c587224224ba1254657b42b091a6c62c39e92/raw/ -> 200, Content-Length 6034, body begins 'Linux kernel\n============' - real original bytes, not a re-encode
2026-10-21	github.com/josephmisiti/awesome-machine-learning/	probe: POST /api/1/origin/save/git/url/ -> 200 {"id":2567968,"save_request_status":"accepted","save_task_status":"pending"} - anonymous, no credential
2026-10-21	withheld	probe: headers on /api/1/origin/.../get/ -> X-Ratelimit-Limit: 120, X-Ratelimit-Remaining: 114
2026-10-21	deposit.softwareheritage.org/1/softwareheritage/	probe: POST (anon multipart) -> 401 (write gated)

More fields (1)

{
 "unit_note": "free_tier.unit is 'mixed' because the headline is denominated in MiB or KiB, which the schema enum does not carry; the verbatim free_tier.stated keeps the true unit. See the storage-dossier handover."
}

Unknown or withheld values (6)

$.evidence[1].url private-url
$.evidence[2].url private-url
$.evidence[4].url private-url
$.free_tier.notes process
$.per_object_cap.notes not-extracted
$.retention.notes not-extracted

software heritage deposit (swh-deposit, the push api) Storage

fails: quotaemail?card?R

deposit.softwareheritage.org/1/softwareheritage/ · rejected-provider · storage/software-heritage-deposit · JSON · compare

Free tier	Upload capped at 100 MiB documented / 200 MiB default in config, and requires HTTP Basic auth - anonymous POST to deposit.softwareheritage.org/1/<collection>/ returns 401. Only useful for so unit none; metered on none
Price	none stated
Per-object cap	not recorded
Retention	not recorded
Egress	not recorded
Identity	email (placeholder) not published
Binding	none no constraint of a named kind was recorded; the verdict rests on the quota sentence rather than on a named limit
Interface	unknown
Tooling	not recorded
Composes with	not recorded
Reliability	Upload capped at 100 MiB documented / 200 MiB default in config, and requires HTTP Basic auth - anonymous POST to deposit.softwareheritage.org/1/<collection>/ returns 401. Only useful for so
Notes	Upload capped at 100 MiB documented / 200 MiB default in config, and requires HTTP Basic auth - anonymous POST to deposit.softwareheritage.org/1/<collection>/ returns 401. Only useful for so
Evidence rung	reported
Checked	2026-10-21 (inherited); stale on 2026-11-04

Evidence (2)

date	source	what it proved
2026-10-21	deposit.softwareheritage.org/1/softwareheritage/	Upload capped at 100 MiB documented / 200 MiB default in config, and requires HTTP Basic auth - anonymous POST to deposit.softwareheritage.org/1/<collection>/ returns 401. Only useful for so
2026-10-21	raw.githubusercontent.com/softwareheritage/swh-deposit/maste	cited as the basis of the rejection: Upload capped at 100 MiB documented / 200 MiB default in config, and requires HTTP Basic auth - anonymous POST to deposit.softwareheritage.org/1/<collection>/ returns 401. Only useful for so

More fields (1)

{
 "dead_end": {
  "failure_class": "quota",
  "why": "Upload capped at 100 MiB documented / 200 MiB default in config, and requires HTTP Basic auth - anonymous POST to deposit.softwareheritage.org/1/<collection>/ returns 401. Only useful for so",
  "retry_when": null,
  "first_recorded": "2026-10-21"
 }
}

Unknown or withheld values (15)

$.chaining not-recorded
$.confidence_note withheld
$.egress.terms not-recorded
$.extras.dead_end.retry_when process
$.free_tier.notes process
$.identity_cost.detail defaulted
$.identity_cost.kind identity-defaulted
$.interface interface-unknown
$.interface_detail withheld
$.limits_note process
$.per_object_cap.notes not-recorded
$.per_object_cap.stated not-recorded
$.retention.notes process
$.retention.stated not-recorded
$.tooling not-recorded