Page MenuHomePhabricator

The Primary Source tool should fill more fields for references
Closed, ResolvedPublic

Description

The Primary Source tool knows more about the source than just the reference url. It should also add "retrieved at" and "stated in".

Having "sourcing circumstances" "Primary Source tool" might also be very valuable for leaving good records about data providence. It would also make Wikidata users more aware of the impact the tool has.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 14 2016, 12:24 PM
Glorian_Yapinus updated the task description. (Show Details)
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.Dec 22 2017, 4:55 PM
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.
Hjfocs closed this task as Resolved.
Hjfocs moved this task from To do to Done on the Wikidata-primary-sources board.
Hjfocs claimed this task.
Hjfocs added a subscriber: Hjfocs.

This depends on the dataset, not on the tool.
StrepHit version 2 has more structured references.

FYI, a folk from my team is developing a script that enriches plain URL references of a dataset serialized in QuickStatements: https://github.com/EdoardoLenzi9/Wikipedia.StrepHit
It does add the stated in and retrieved references.
https://phabricator.wikimedia.org/T148148
phabricator.wikimedia.org
crawl=CC-MAIN-2019-26
53815
application/http; msgtype=response
sha1:5W3KRKG7EIJ42BQS2RLN2EVL2SAAXYIQ
<urn:uuid:822f594b-5177-4a35-b196-e30dbe8be1e0>
2019-06-24T09:05:38Z
208.80.154.224
text/html
sha1:MZ3OH2CQ6RABSLCOIRHEO6YQINUAXZWG
<urn:uuid:5598a02d-7576-4757-89b6-8d6b1062cb68>
https://phabricator.wikimedia.org/T148148
response
<urn:uuid:f31b946b-8269-4976-b723-7e41db471d60>
isPartOf: CC-MAIN-2019-26
publisher: Common Crawl
description: Wide crawl of the web for June 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-93-203-79.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.1-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/