fshy

fshy / Debrid

PubMed Central Open Access Subset

pmc.ncbi.nlm.nih.gov · Debrid · 1 record · JSON

PubMed Central Open Access Subset (pmc-oa-opendata on S3) Debrid

★ qualifiesanonymousno cardM

pmc.ncbi.nlm.nih.gov/tools/pmcaws/ · open-access full-text research corpus (world-readable S3 bucket) · debrid/pmc-oa-opendata · JSON · compare

Free tier'PMC makes the datasets described below freely accessible on Amazon Web Services (AWS), without charge, through either an HTTPS or S3 URL, and without any log-in requirement for retrieval.' The README reports 'roughly 8 million PMC article versions'.
unit unlimited; metered on none; The bucket is s3://pmc-oa-opendata in us-east-1; the AWS Open Data Sponsorship Program covers the transfer.
PriceUSD 0
Per-object capnot applicable: one object per article file (JATS XML, plain text, PDF, media). A full-text XML measured here was 109,974 bytes.
Retentionarticles stay while their licence holds; 'If a version is removed from the S3 bucket, its prefix and all associated files are deleted from the bucket with no explicit notification. Inventory reconciliation is the only supported way to detect removals.' The inventory is regenerated daily and kept 30 days.
EgressFree and unmetered: NLM pays through the AWS Open Data Sponsorship Program. 'No charges, usage fees or royalties are paid to NLM for these data.' Both https://pmc-oa-opendata.s3.amazonaws.com/<prefix>/<file> and s3:// with --no-sign-request work.
free_allowance: unlimited, sponsored
Identityanonymous 'without any log-in requirement for retrieval'; an unsigned HTTPS GET for a full-text XML returned 200 here. No Earthdata-style account, no API key.
Limits
  • request-rate no request cap is published for the S3 bucket; the article-dataset page restricts automated retrieval to the Cloud Service, OAI-PMH, E-utilities and BioC API source
  • per-object-cap one object per article file; measured full-text XML is 109,974 bytes, so a 1 TiB pull is roughly 10 million objects
The 2026-08-24 change removed the legacy FTP and cloud dataset files and left the S3 bucket as the distribution path
Bindingnone Nothing fails at 1 TiB/month: the whole corpus is smaller than that. At the measured 109,974 bytes per article version, 8 million versions is about 880 GB of XML before PDFs and media - arithmetic from the two published numbers, not a measurement of the bucket.
Interfaces3, rest, cli S3 REST and HTTPS on the same bucket. aws s3 cp --no-sign-request s3://pmc-oa-opendata/<PMCID>.<version>/<PMCID>.<version>.xml, or curl the equivalent https URL. S3 Inventory under inventory-reports/ gives a daily CSV manifest.
endpoint: https://pmc-oa-opendata.s3.amazonaws.com/
ToolingAWS CLI with --no-sign-request, curl, boto3 unsigned, rclone (S3 remote with anonymous access). Any of them reaches the bucket; nothing needs an account.
Composes withThe full-text corpus that pairs with Europe PMC's metadata index: harvest PMID/PMCID lists from Europe PMC or the PMC inventory, then pull the XML here. Licences per article decide what a derivative may redistribute.
ReliabilityRemoved articles vanish with no notification; the inventory lags the bucket by up to a day, so a manifest-driven sync always races deletions.
NotesThis is the strongest no-account research corpus: 8 million article versions, no login, S3 range reads, a daily inventory. The 2026-08-24 distribution change is the reason a record written before it would name FTP paths that no longer exist.
Evidence rungmeasured Two objects were fetched anonymously and hashed here (README.txt 14,524 bytes; one full-text XML 109,974 bytes) and the bucket's accept-ranges is byte. What is not measured is aggregate throughput at 1 and 8 connections.
Checked2026-10-24 (live); stale on 2026-11-23
Parsed numbers
  • $.limits[1].value limit:per-object-cap 1.0 TiB
  • $.price_per_tb_month price-per-tb USD0.0 TB / month
Evidence (6)
datesourcewhat it proved
2026-10-23pmc.ncbi.nlm.nih.gov/tools/pmcaws/world-readable bucket s3://pmc-oa-opendata, 'without any log-in requirement for retrieval', --no-sign-request, and the removal-without-notification clause
2026-10-23pmc-oa-opendata.s3.amazonaws.com/README.txt'roughly 8 million PMC article versions' and the <PMCID>.<version>/ object layout
2026-10-23pmc.ncbi.nlm.nih.gov/tools/cloud/'All legacy PMC Article Dataset files on the FTP and Cloud Services were removed the week of August 24, 2026' - the bulk path changed in 2026
2026-10-23pmc.ncbi.nlm.nih.gov/tools/openftlist/only the Cloud Service, OAI-PMH, E-Utilities and BioC API may be used for automated retrieval of PMC content
2026-10-23probe:https://pmc-oa-opendata.s3.amazonaws.com/README.txt200, 14,524 bytes, sha256 257a001b400485176702ed49ca7607d73cd83c581bffb6dd3c9be127a2e42634, anonymous
2026-10-23probe:https://pmc-oa-opendata.s3.amazonaws.com/PMC10009402.1/PMC10009402.1.xml200, 109,974 bytes, sha256 75d9e0ab27e80e8660296e33bb1832dd8237080d5782b229bbd81fb4919c323a, accept-ranges: bytes
More fields (5)
{
 "bucket": "pmc-oa-opendata",
 "region": "us-east-1",
 "article_versions": 8000000,
 "xml_sha256": "75d9e0ab27e80e8660296e33bb1832dd8237080d5782b229bbd81fb4919c323a",
 "estimated_corpus_gib": 819
}