Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Characterization Services

Access Services

Common Tools

  • Anvl ANVL : A Simple Record Syntax
  • Checkm: A Checksum-based Manifest FormatCutie: A Simple Directory-based Queuing Service
  • Datenorm: Date normalization
  • ERC : Electronic Resource Citation and Kernel Metadata
  • LockIt: A Simple File-based Convention for Resource Locking
  • Namaste: Directory Description with Namaste Tags
  • RUU: (Are You You?): User account authentication
  • THUMP: The HTTP URL Mapping Protocol

...
https://confluence.ucop.edu/pages/diffpages.action?originalId=140771360&pageId=140771362
confluence.ucop.edu
crawl=CC-MAIN-2020-29
52219
application/http; msgtype=response
sha1:HMZIMWV4I4QNT3UNUC32UIHPUB3VU4XX
<urn:uuid:134ff97b-5be8-429f-8096-f092b09a25b8>
2020-07-02T05:33:13Z
128.48.70.55
text/html
sha1:UHU2FFTM6XX7XJT6FFWSZGAY7VGSRCHR
<urn:uuid:c3453fee-bf60-4a69-8da0-0cef720542c4>
https://confluence.ucop.edu/pages/diffpages.action?originalId=140771360&pageId=140771362
response
<urn:uuid:20886081-62aa-470d-ae4b-ff9d0a80ff7b>
isPartOf: CC-MAIN-2020-29
publisher: Common Crawl
description: Wide crawl of the web for July 2020
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-34.ec2.internal
software: Apache Nutch 1.17 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/