December 2018
Developments and innovations to Goobi workflow in December 2018

Extended support of standard data via DANTE

In addition to the metadata fields for texttareas, select menus, read-only fields or the connection of GeoNames and GND, which have been available for a long time, the GBV DANTE standard data service has been connected. Dante currently provides about 90 different vocabularies which can all be used with this revised implementation.
To use the DANTE API the file goobi_metadataDisplayRules.xml must be extended by the following configuration:
goobi_metadataDisplayRules.xml
1
<!-- dante configuration for the metadata field DocLanguage-->
2
<dante ref="DocLanguage">
3
<!-- name of the vocabulary, a list of all vocabularies can be found here: http://api.dante.gbv.de/voc, the vocabulary name can be found inside of the element notation -->
4
<source>languages_gnd</source>
5
<!-- the list of fields to import the value from. The list is ordered and the first value that was found will be used for the display. Can be overwritten inside of the UI -->
6
<field>NORM_LABEL_de, NORM_LABEL_en, NORM_LABEL_fr, NORM_LABEL_es</field>
7
</dante>
Copied!
The field configured in each case is then available in the METS editor. The search form is opened by means of the magnifying glass icon so that the desired value can be entered within the selected standard data vocabulary. The desired hit can then be selected from the displayed search hit list. Within the desired hit, the value to be imported can also be selected from a selection list and confirmed by clicking on the green button.
Overlapping the Dante search within the METS editor

Correction of the determination of directory sizes when using S3

The integration of Amazon S3 as a storage system for storing large amounts of data has already made massive progress in recent months. During this work, further improvements and error corrections were always necessary. In addition to a reduction of file accesses, the determination of directory sizes, which did not always provide correct results when using S3, has also been revised.

New plugin for importing province data from the GBV catalogue

If province data have already been entered in the GBV catalogue, they can now be imported as well. Since this metadata is not contained within the MARC record, it cannot be imported during the initial import of the metadata. The import therefore takes place in a second step. For this purpose, the EPN of the A-record is used to search for the corresponding Pica+ record. The system then checks whether provenance data is available for the copy used. This check takes place by matching the EPN in field 092B$2 and the library symbol in field 092B$1.
To be able to use the plugin, a metadata group Provenience must exist in the rule record:
ruleset.xml
1
<MetadataType>
2
<Name>ProvenienceCode</Name>
3
<language name="de">Code</language>
4
<language name="en">ProvenienceCode</language>
5
</MetadataType>
6
<MetadataType normdata="true">
7
<Name>ProveniencePrevOwner</Name>
8
<language name="de">Besitzer</language>
9
<language name="en">ProveniencePrevOwner</language>
10
</MetadataType>
11
<MetadataType>
12
<Name>ProvenienceCharacteristic</Name>
13
<language name="de">Characteristik</language>
14
<language name="en">ProvenienceCharacteristic</language>
15
</MetadataType>
16
<MetadataType>
17
<Name>ProvenienceDate</Name>
18
<language name="de">Datum</language>
19
<language name="en">ProvenienceDate</language>
20
</MetadataType>
21
<MetadataType>
22
<Name>ProvenienceExplanation</Name>
23
<language name="de">Erläuterung</language>
24
<language name="en">ProvenienceExplanation</language>
25
</MetadataType>
26
<MetadataType>
27
<Name>ProvenienceGND</Name>
28
<language name="de">GND</language>
29
<language name="en">ProvenienceGND</language>
30
</MetadataType>
31
<Group>
32
<Name>Provenience</Name>
33
<language name="de">Provenienz</language>
34
<language name="en">Provenience</language>
35
<metadata>ProvenienceCode</metadata>
36
<metadata>ProveniencePrevOwner</metadata>
37
<metadata>ProvenienceCharacteristic</metadata>
38
<metadata>ProvenienceDate</metadata>
39
<metadata>ProvenienceExplanation</metadata>
40
<metadata>ProvenienceGND</metadata>
41
</Group>
Copied!
Further adjustments are not necessary. The plugin itself is part of the collection goobi-plugin-opac-marc.jar and can be accessed via the name HaabProvenienceOpac. An activation within the configuration file goobi_opac.xml is not possible, because the plugin only delivers the provenance metadata and no structural elements are created. To integrate it into the workflow as a step after the initial import, the step plug-in intranda_step_provenience_request from the goobi-plugin-step-catalogue-request repository is also required.

Import EAD records directly into Goobi

Goobi is now able to import individual records from EAD find books. To do this, the EAD data records available in XML are periodically imported into the BaseX XML database. From now on, this database serves as a dynamically updated EAD memory and can be queried with the help of a special OPAC plug-in via the RESTXQ query language instead of a typical library catalog.
The special feature compared to a normal XML-based import is that the tectonics of the find book are preserved, since they are also stored within the XML database. If the tectonics is changed and the previously imported data record is moved to another structure, then only the EAD file must be read into the XML database in updated form. Since the identifiers of the data records are retained, presentation software such as the Goobi viewer, which also has access to this EAD memory, can query the current status of the tectonics live.
Further detailed information about the implementation including installation instructions can be found here:

Display metadata within tasks and in task details

Goobi workflow has been enhanced to allow you to display specific metadata from the METS file within individual accepted tasks as well as properties.
The display of this metadata integrates itself into the range of properties that can also be displayed and edited and can be individually configured for each task of individual workflows. Such a configuration looks like the following within the file goobi_processProperties.xml:
goobi_processProperties.xml
1
<metadata name="CatalogIDDigital">
2
<project>*</project>
3
<showStep name="Print docket" />
4
</metadata>
5
<metadata name="DocLanguage">
6
<project>*</project>
7
<showStep name="Print docket" />
8
</metadata>
9
<metadata name="TitleDocMain">
10
<project>*</project>
11
<showStep name="Print docket" />
12
</metadata>
Copied!
The display of descriptive metadata is also possible within the process details. For this purpose, a new area has been created below the tasks that lists these metadata.
Display of selected metadata within the task details
With the help of this new functionality and future developments, redundant storage of some metadata will become superfluous in the future. From now on, information that is currently still available as a property as well as within the METS file can only be stored in the METS file and read from there. It is important that the checkbox Update metadata index when finishing is activated within the individual tasks of the workflow to ensure efficient caching of the data from the METS file for display.
Configuration for updating the cache for information from the METS file
For a complete update of the cache for the fast display of this metadata within the web interface, the existing GoobiScript updateDatabaseCache can of course also be called for several processes simultaneously.
Updating the database cache for multiple operations using GoobiScript
https://docs.goobi.io/goobi-workflow-digests-en/2018/12
docs.goobi.io
crawl=CC-MAIN-2022-21
618793
application/http; msgtype=response
sha1:U34S2FOGQ3D4ZEI64WMNGFENG5LYHGW6
<urn:uuid:ba974c17-04b3-4d08-9972-07bec419b40d>
2022-05-22T14:44:03Z
104.18.0.145
text/html
sha1:OCHFJJTP4YGKO4CRMPZMUCYCJ6VZLTWX
<urn:uuid:4e276fd1-2a4f-4c2a-91c8-5cb5cb1d5d78>
https://docs.goobi.io/goobi-workflow-digests-en/2018/12
response
<urn:uuid:b3aafaff-5584-4be3-97c0-36e930deb2a3>
isPartOf: CC-MAIN-2022-21
publisher: Common Crawl
description: Wide crawl of the web for May 2022
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-215
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.3-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/