Groovy HTML CSS JavaScript XSLT Java Other
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
SyncClientLib
aggregation/marc_import
assets
automation/science_direct
deployment
deps/elasticsearch-client-groovy/1.0.1
describe
docker
gdocs/docs
images
interface_docs
linkedData
projectDocs/wip
refine
reporting
scripts
server/gokb
spikes
testdata
utils
.gitignore
DEVELOPER.md
DOCKER.md
LICENSE.txt
README.md
RELEASE-NOTES.txt
post-install.sql

README.md

gokb

GOKB is an environment for managing the collaborative import, cleaning and maintaining of publisher title lists. The focus is the supply of information around the administration and availability of print and electronic resources rather than specific bibliographic information. The goal of GOKb is to reduce the burden of data maintenance and editing on any one party by sharing the load over a collaborative network of subject matter experts. Because the information is global, and the same for everyone, the idea is to focus all our effort on editing one copy of the information that can then be freely reused and shared.

Some screenshots follow

Dashboard File Upload Login Master List Perms Title Full Titles Side-By-Side ToDos Welcome

Development Information

  • To build the refine extension, ** cd into gokb-phase1/refine/extensions/gokb

  • run "ant" with any local overrides for refine installation dir and tomcat for server jars, eg ** ant -Drefine.dir=/home/ibbo/google-refine-2.5 -Dserver.dir=/home/ibbo/apache-tomcat

Copy (Or arrange to symlink) the gokb directory to your local equivalent of ~/google-refine-2.5/webapp/extensions/

I packaged up the gokb extension with #

zip -r gokb.zip ./gokb

Server side component (dev)

from gokb-phase1/server/gokb grails run-app

First run

Orgs Import

A script is provdied to import the ncsu orgs list, run from gokb-phase1/scripts/import/orgs as follows:

./ncsu_orgs_imp.groovy ./ncsu-auth-orgs-roles-2013-01-11.csv

For parsing out Platform URLs http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/net/InternetDomainName.html System.out.println(InternetDomainName.fromLenient(uriHost).topPrivateDomain().name());
https://github.com/k-int/gokb-phase1
github.com
crawl=CC-MAIN-2018-34
147204
application/http; msgtype=response
sha1:FVYYW3C6WNE6DUMMTLI56AVHGTDEZ46C
<urn:uuid:3e312c3b-5dc2-41f3-925c-209e84f2ec9b>
2018-08-19T10:34:37Z
192.30.253.112
text/html
sha1:7KIO6WEOVBH2RKVMLELUNPWXEARHDCNO
<urn:uuid:d8bb3764-2c6d-4eb3-a252-48a74242d946>
https://github.com/k-int/gokb-phase1
response
<urn:uuid:37ba4a43-09d2-43e3-8a9d-0b59c44e8076>
isPartOf: CC-MAIN-2018-34
publisher: Common Crawl
description: Wide crawl of the web for August 2018
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-199-169.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 0.11-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/