Datasets ▶ Sci-Hub
If you are interested in mirroring this dataset for archival or LLM training purposes, please contact us.

For a background on Sci-Hub, please refer to its official website, Wikipedia page, and this podcast interview.

Note that Sci-Hub has been frozen since 2021. It was frozen before, but in 2021 a few million papers were added. Still, some limited number of papers get added to the Libgen “scimag” collections, though not enough to warrant new bulk torrents.

We use the Sci-Hub metadata as provided by Libgen.li in its “scimag” collection. We also use the dois-2022-02-12.7z dataset.

Resources
https://annas-archive.gs/datasets/scihub
annas-archive.gs
CC-MAIN-2024-10
44555
application/http; msgtype=response
sha1:TKHAQQLIAIKCC2SOVA27RQTRTCMWZVNQ
<urn:uuid:4c58a1e6-991a-432e-9e50-3d716acbdba6>
2024-03-04T08:23:58Z
172.67.169.27
text/html
sha1:SLULEVXMLOYTRHEOGZKACAVP32HU2NCJ
<urn:uuid:e4dcd090-00e3-4434-bf17-0e1004e412a2>
https://annas-archive.gs/datasets/scihub
response
<urn:uuid:c2db8dd3-d3b3-4f6a-b34f-ca47c1fac524>
isPartOf: CC-MAIN-2024-10
publisher: Common Crawl
description: Wide crawl of the web for February/March 2024
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-169
software: Apache Nutch 1.19 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.5-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/