Bibtex export, problem with LaTex control chars

A place for users to ask each other questions, make suggestions, and discuss Bookends.
Post Reply
mwra
Posts: 48
Joined: Mon Jun 15, 2015 7:01 am

Bibtex export, problem with LaTex control chars

Post by mwra » Tue Oct 16, 2018 7:29 am

I'm exporting from BE 13.1.4 to Bibtex format and am hitting several issues when using that data with pdflatex & biber in the 2018 MacTex distro. After much to-and-fro the problem appears to be linked to unescaped ampersands (and possible also percent signs) in BE's output Bibtex. I've been back and forth but can't find a preference or biblio setting to remedy this except by manually post-processing the output Bibtex file.

FWIW, I'm exporting in UTF-8 encoding which happily copes with thinks like diacriticals in names without needing additional escaping. My hunch is that for pre-emptive 'safe' LaTeX handing the following should be escaped regardless in Bibtex export: octothorpe (#), circumflex/caret (^), dollar sign ($), Precent symbol (%), ampersand (&), (literal) curly braces ( { } ), underscore (_).

I'm also having a problem with indicating that an 'author' that is a company name, e.g. "Cunningham & Cunningham, Inc." should be parsed as a single literal string, other wise I get a reference to things like,

Code: Select all

Cunningham, Cunningham &, I.
. At present I'm post-editing the output to add extra { } around the name in the author field. This exports OK but means the item now sorts incorrectly as the first character of the author name is read as '{'. Also, there's scope for the curly braces being misunderstood as literals and escaped (see above).

Jon
Site Admin
Posts: 8003
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bibtex export, problem with LaTex control chars

Post by Jon » Tue Oct 16, 2018 9:25 am

For problem #2, in Bookends end the name with a comma and Bookends will treat it as an institutional name:

Cunningham & Cunningham, Inc.,

Jon
Sonny Software

mwra
Posts: 48
Joined: Mon Jun 15, 2015 7:01 am

Re: Bibtex export, problem with LaTex control chars

Post by mwra » Tue Oct 16, 2018 11:01 am

Thanks. However, the author string "Cunningham & Cunningham, Inc." and the author string "Cunningham & Cunningham, Inc.," generate the same Bibtex output line:

Code: Select all

author = {Cunningham & Cunningham, Inc.}, 
By comparison, "Microsoft Corporation," does modify the Bibtex. Thus it seems the problem is the (literal) comma in the first company's name. It seems the format "Company Name, Inc." is quite common in the US. For Web citation - something many established citation formats describe poorly if at all - there may be no single human author therefore it is essentially the name of the org running the site which brings us back to problem items like "Company Name, Inc.".

I guess a short term hack is to omit the inline comma in the name. Then again some of my colleagues would flag that as a proof reading error when checking references. As I only use Bibtex export at present another kludge might be to put a LaTeX backslash before the inline comma, e.g. "Company Name\, Inc.", though this would screw up exports in other formats.

Jon
Site Admin
Posts: 8003
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bibtex export, problem with LaTex control chars

Post by Jon » Tue Oct 16, 2018 11:15 am

I guess I don't understand what you want. If you put a comma after a name, Bookends will output the contents without modification of any kind (i.e. no surname first or last rearrangements, no removal of punctuation).

Jon
Sonny Software

mwra
Posts: 48
Joined: Mon Jun 15, 2015 7:01 am

Re: Bibtex export, problem with LaTex control chars

Post by mwra » Tue Oct 16, 2018 11:19 am

Yes, but if the name already includes a comma e.g. the first example in my last reply, adding an extra comma at the end doesn't help as Biber then sees the original inline comma and 'parses' the name.

The manual post edit is from this

Code: Select all

author = {Cunningham & Cunningham, Inc.}, 
to this:

Code: Select all

author = {{Cunningham & Cunningham, Inc.}}, 
I as looking for someway to trigger the latter. Does that explain the problem better? Thanks, I do realise this is an odd edge case.

Jon
Site Admin
Posts: 8003
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bibtex export, problem with LaTex control chars

Post by Jon » Tue Oct 16, 2018 1:09 pm

I see. Bookends won't add the extra brackets around the name for you, but you can do that yourself in the reference when you know it's an institutional name destined for BibTeX export

Jon
Sonny Software

mwra
Posts: 48
Joined: Mon Jun 15, 2015 7:01 am

Re: Bibtex export, problem with LaTex control chars

Post by mwra » Tue Oct 16, 2018 1:14 pm

Might it therefore be possible for sort in the main window list to ignore { and } when storing. Otherwise brace enclosed authors sort out of sequence. Thanks.

Jon
Site Admin
Posts: 8003
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bibtex export, problem with LaTex control chars

Post by Jon » Tue Oct 16, 2018 1:21 pm

No, the reference list sort is alphanumeric, you can't tell it to ignore any arbitrary leading characters.

FWIW, even though it sorts out of order, if you so a search for Cunningham (in this case) Bookends will find and display it.

Jon
Sonny Software

Jon
Site Admin
Posts: 8003
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bibtex export, problem with LaTex control chars

Post by Jon » Tue Oct 16, 2018 2:29 pm

mwra wrote:
Tue Oct 16, 2018 7:29 am
FWIW, I'm exporting in UTF-8 encoding which happily copes with thinks like diacriticals in names without needing additional escaping. My hunch is that for pre-emptive 'safe' LaTeX handing the following should be escaped regardless in Bibtex export: octothorpe (#), circumflex/caret (^), dollar sign ($), Precent symbol (%), ampersand (&), (literal) curly braces ( { } ), underscore (_).
Those characters will be escaped if you tell Bookends to generate the bibliography as BibTeX Markup.

Jon
Sonny Software

mwra
Posts: 48
Joined: Mon Jun 15, 2015 7:01 am

Re: Bibtex export, problem with LaTex control chars

Post by mwra » Wed Oct 17, 2018 11:56 am

In BE Prefs/BiBTeX I have all 3 tick-boxes set in the right column. I presume it is the second one that does escaping. Oddly, despite that, in both the formatted preview and the Format manager for Bibtex, no encoding of diacriticals occurs. I see:

Code: Select all

address = {Montréal, Québec, Canada Republic and Canton of Geneva, Switzerland}, 
Inline use of ampersand, percent and dollar signs are also not encoded (for the abstract examples I've pruned some text):
unescaped %

Code: Select all

abstract = {Over 85% of the user’s interaction is direct—a single point-and-click designates both object and operation.},

unescaped $

Code: Select all

abstract = {Instead the promise of exponential growth in computing/$/volume demanded that ...},

unescaped &

Code: Select all

volume = {Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing},
Aha, the problem is the Bibliography dialog which allows you to set the BibTex filter but invalid formats. I'd left the output format as 'UTF8', assuming this would give me a UTF-8 text file in BibTeX format. It does, sort of , in terms of adding the main { } around the output field data but not doing the LaTeX style escaping. IOW, to the casual glance it *looks* like BIBTeX but isn't really. If The export filter is BiBTex, I'd suggest that BiBTeX be auto-selected as the format choice. Why would one want to export incompletely encoded BiBTeX. The manual, p.325 doesn't really explain this.

I can confirm that with the Biblio export set to use the BiBTeX filter and export to disk and with the 'BibTeX Markup' format selected I do get a clean export.

So, the hard-to guess villain of the piece here was the Biblio export allowing me to select an incompatible export format choice.

FWIW, 'BibTeX Markup' outputs a UTF-8 [sic] text file with a '.bib' extension.

joao
Posts: 25
Joined: Fri Jun 17, 2016 4:23 am

Re: Bibtex export, problem with LaTex control chars

Post by joao » Fri Jan 11, 2019 12:38 pm

Jon wrote:
Tue Oct 16, 2018 2:29 pm
Those characters will be escaped if you tell Bookends to generate the bibliography as BibTeX Markup.
Hi Jon.
Is there a way we can tell Bookends to do this (escape the characters) via applescript?
I can see in the user guide that we can choose "rrtf" or "text" class when exporting, but no mention of bibtex.

Joao

Jon
Site Admin
Posts: 8003
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bibtex export, problem with LaTex control chars

Post by Jon » Fri Jan 11, 2019 12:47 pm

No, but you can in the application itself (Biblio -> Bibliography). Or do you need to use AppleScript?

Jon
Sonny Software

joao
Posts: 25
Joined: Fri Jun 17, 2016 4:23 am

Re: Bibtex export, problem with LaTex control chars

Post by joao » Fri Jan 11, 2019 3:06 pm

Don't need to use applescript, its just more efficient to update an existing bibtex library that way.
I did not know about the Biblio menu export (was going via the file menu). That is a bit faster but still slow.

The problem with doing it via the application (whether through the Biblio or File menu), is that it takes too many clicks:
It doesn't accept selections, so I have to select my group, select all the references in the group, right click to 'mark selected' so they become hits, and only then can I actually use the export feature from the menu.
If its too much to get the functionality via applescript, it would be great if both menu actions above could accept straight selections from the window rather than demanding hits. It would be even better if they interpreted a selected group (with the exception of 'All') without any active selections as the user wanting to export all the records in the group.

That would make the export process a bit easier. I realise that in most apps 'export' is always seen as a way of escaping a walled garden, but in the case of bibtex exports, its more about functionality.

Jon
Site Admin
Posts: 8003
Joined: Tue Jul 13, 2004 6:27 pm
Location: Bethesda, MD
Contact:

Re: Bibtex export, problem with LaTex control chars

Post by Jon » Fri Jan 11, 2019 3:50 pm

You're doing it the hard way. Show the group and do one of these:

1. Click on the checkbox of ONE reference with the Shift key held down. Now they're all marked. That's two clicks.

2. Right click on the reference list and select Mark All. That's one click and one right-click.

Jon
Sonny Software

joao
Posts: 25
Joined: Fri Jun 17, 2016 4:23 am

Re: Bibtex export, problem with LaTex control chars

Post by joao » Fri Jan 11, 2019 4:15 pm

Thanks.The first option works great.
Didn't realise there were simpler ways....

Post Reply
https://www.sonnysoftware.com/phpBB3/viewtopic.php?f=2&p=21755
www.sonnysoftware.com
crawl=CC-MAIN-2019-13
66830
application/http; msgtype=response
sha1:PISESZAIQFWTCH5H3G2OHGMNYRUFVGNA
<urn:uuid:829fba06-c885-492d-820e-8d7d58f3be23>
2019-03-23T07:11:31Z
216.194.196.10
text/html
sha1:HMNJHT23CPGKWOS7BEDOD6UDTSJMVWA6
<urn:uuid:49615ce5-174f-4e71-9a55-a63e4c927f2a>
https://www.sonnysoftware.com/phpBB3/viewtopic.php?f=2&p=21755
response
<urn:uuid:a6cfcbf6-6d58-466b-9257-0789d3337b73>
isPartOf: CC-MAIN-2019-13
publisher: Common Crawl
description: Wide crawl of the web for March 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-145-45-50.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 0.11-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/