Subfield order in bib. record changes when editing authority

Bug #712490 reported by Marjolein Kremer on 2011-02-03
38
This bug affects 8 people
Affects Status Importance Assigned to Milestone
Evergreen
Medium
Unassigned
3.1
Undecided
Unassigned
3.2
Undecided
Unassigned

Bug Description

Release 2.0 RC2:
After editing an authority the order of the subfield change in the linked bibliografic record.
Example:
Step 1:
Create a bib record, from field 100: create authority immediately
100 $a Opland $e ontwerper $0 (NL-AMISG)78
Validate and save bib record.
Step 2:
Manage authorities, Edit authority
100 $a Opland, (we added a comma)
Step 3:
Searching the catalog for the linked bib record
100 $0 (NL-AMISG)78 $a Opland, $e ontwerper
Is it normal that the $0 changes position?

This has been the normal behaviour since the summer of 2010 when it
was initially implemented. As the content of the $0 subfield is not
expected to be read by any humans other than to cataloguers, we didn't
think this presented a problem.

Tools that display the contents of a 650, for example, should strip
the $0 subfield out no matter where it appears in the field, right?

Dan Scott (denials) wrote :

I've now confirmed that this is a problem for subfields that are not part of the list of controlled subfields. For example, in the current trunk mapping of controlled fields/subfields, 600$c is not controlled. Here's what happens when you control the 600 field containing a $c, then modify the linked authority record:

Original, uncontrolled field:

=600 00 $aWilliam,$cPrince, grandson of Elizabeth II, Queen of Great Britain,$d1982-$xRelations with women.

Field after it has been controlled and the $0 has been added:

=600 00 $aWilliam,$cPrince, grandson of Elizabeth II, Queen of Great Britain,$d1982-$xRelations with women.$0(CONS)2

Field after the linked authority record has been modified:

=600 00$cPrince, grandson of Elizabeth II, Queen of Great Britain,$0(CONS)2$aWilliam,$d1982-$xRelations with women.

As the order of subfields other than $0 does matter for display, even for uncontrolled subfields, this is a problem.

Changed in evergreen:
status: New → Confirmed
Jason Stephenson (jstephenson) wrote :

Reported against 2.0, where it will NOT be fixed.

I am setting to incomplete pending confirmation that it is still an issue that needs fixing in 2.1+.

Changed in evergreen:
status: Confirmed → Incomplete
Yamil (ysuarez) wrote :

I saw this bug occur after we would merge two duplicate authority records that had bibs linked to at least one of the auth records (or both of them). This was a couple of years ago, at every EG conference since 2012 someone mentions that bug to me. Good to know that there was a an existing LP bug for it.

tags: added: authority cataloging
Dan Wells (dbw2) wrote :

I can confirm that this bug still happens exactly as diagnosed by Dan S. in comment #2. This was tested on a relatively recent version of master.

Changed in evergreen:
status: Incomplete → Confirmed
importance: Undecided → High
Jason Stephenson (jstephenson) wrote :

Given that this has not been targeted at any milestones, nor is anyone working on it AFAIK, I am setting importance to Medium.

Changed in evergreen:
importance: High → Medium
no longer affects: evergreen/2.7
Bill Erickson (berick) on 2016-10-21
no longer affects: evergreen/2.8
Bill Erickson (berick) wrote :

If I'm reading the code right, the problem appears to be that the DB function vandelay.replace_field() performs the replacement action in 2 steps, one to remove the controlled subfields, followed by another to add them back with the new values. Since the add action is decoupled from the remove action, the controlled subfields are simply added to the end of the marc field.

Presumably vandelay.replace_field() will need to morph into something a little smarter, like vandelay.strip_field() or vandelay.add_field(), which parse the record in search of data to affect.

I think it might also be possible to extend vandelay.strip_field() and teach it to apply a replacement value to each would-be-deleted subfield instead of deleting them.

Bill Erickson (berick) on 2017-10-18
Changed in evergreen:
assignee: nobody → Bill Erickson (berick)
Bill Erickson (berick) wrote :

http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/lp712490-vandelay-replace-field-retain-order

This branch replaces vandelay.replace_field() function with a PGPERL handler, very similar to vandelay.strip_field() and vandelay.add_field(). In short, when replacing subfield-level values, it performs a merge instead of a delete+add.

The code iterates each affected field, leaving uncontrolled subfields alone, replacing values for matching controlled subfields, splicing in new controlled subfields (i.e. those that only existing on the source record), and deleting controlled subfields that only exist on the target field.

The final result is consistent with the existing vandelay.add_field() with respect to which fields are present. The only difference is the field order.

I have confirmed Dan's example from comment #2 now retains the field order (assuming 600$c is uncontrolled).

Includes PGTAP test.

To test:

[1] Link a an authority record to a bib field by adding a $0 as the final subfield.
[2] Modify the authority record
[3] Confirm the modifications propagated to the affected bib field leaving the $0 as the last field.

tags: added: pullrequest
Changed in evergreen:
milestone: none → 3.0.1
assignee: Bill Erickson (berick) → nobody
Changed in evergreen:
milestone: 3.0.1 → 3.0.2
Changed in evergreen:
milestone: 3.0.2 → 3.0.3
Changed in evergreen:
milestone: 3.0.3 → 3.0.4
Changed in evergreen:
milestone: 3.0.4 → 3.0.5
Changed in evergreen:
milestone: 3.0.5 → 3.0.6
Changed in evergreen:
milestone: 3.0.6 → 3.0.7
Changed in evergreen:
milestone: 3.0.7 → 3.0.8
Changed in evergreen:
milestone: 3.0.8 → 3.1.3
Changed in evergreen:
milestone: 3.1.3 → 3.1.4
Changed in evergreen:
milestone: 3.1.4 → 3.1.5
no longer affects: evergreen/2.12
Changed in evergreen:
milestone: 3.1.5 → 3.1.6
Meg Stroup (mstroup) wrote :

Followed process outlined in #8, using same example/subfields from #2.

Modifying authority record now leaves $0 as final subfield. Authority displays correctly in OPAC view.

Changed in evergreen:
milestone: 3.1.6 → 3.2.1
Jane Sandberg (sandbej) wrote :

I agree with Meg. With Bill's fix, editing the authority record doesn't cause the uncontrolled subfields to get out of order. Thanks, Bill and Meg! It will be very nice to have this fixed.

I have tested this code and consent to signing off on it with my name, Jane Sandberg, and my email address, <email address hidden>.

tags: added: signedoff
Changed in evergreen:
milestone: 3.2.1 → 3.2.2
Changed in evergreen:
milestone: 3.2.2 → 3.2.3
Changed in evergreen:
milestone: 3.2.3 → 3.3-beta1
status: Confirmed → New
Dan Wells (dbw2) on 2019-02-06
Changed in evergreen:
assignee: nobody → Dan Wells (dbw2)
Changed in evergreen:
milestone: 3.3-beta1 → 3.3-rc
Changed in evergreen:
assignee: Dan Wells (dbw2) → Jason Stephenson (jstephenson)
Jason Stephenson (jstephenson) wrote :

Pushed to master, rel_3_2, and rel_3_1.

Thanks, everyone!

Changed in evergreen:
status: New → Fix Committed
no longer affects: evergreen/3.0
Changed in evergreen:
assignee: Jason Stephenson (jstephenson) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers
https://bugs.launchpad.net/evergreen/+bug/712490
bugs.launchpad.net
crawl=CC-MAIN-2019-13
86948
application/http; msgtype=response
sha1:LDC5SY36WYNAOZWXKJYP3LWOY5OPAYYI
<urn:uuid:84fe1afe-15e9-4c47-89f3-d9f91a7d7742>
2019-03-19T03:40:38Z
91.189.89.225
application/xhtml+xml
sha1:NOQG6JQCBNF4227X4MIUIUSFK3VTVOX6
<urn:uuid:d752c366-0e0b-4aad-85c5-8644ea604f67>
https://bugs.launchpad.net/evergreen/+bug/712490
response
<urn:uuid:f93fd633-fb49-4bf7-9784-d1f62bca49eb>
isPartOf: CC-MAIN-2019-13
publisher: Common Crawl
description: Wide crawl of the web for March 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-5-254-60.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 0.11-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/