Stemming Dictionaries - Fluid Topics - 3.2 - Technical Notes

Understand Linguistics in Fluid Topics

Product
Fluid Topics
FT_Version
3.2
Category
Technical Notes
audience
public

Stemming dictionaries can only include one-word entries. Stemming is not applied correctly if altLabel (i.e. synonyms) or prefLabel (i.e. main term) are composed of multiple terms.

Entries including special characters, such as hyphenated forms of a word or dates, like anti-biotics or 25/jan/2014, are considered multi-term entries. As a consequence, such entries are invalid in a stemming dictionary.

Examples of invalid entries
  • 25 janvier 2014 = 25/jan/2014
  • anti-biotics = antibiotics
Example of a valid entry
  • Français = Françaises
https://doc.antidot.net/r/3.2_public/Understand-Linguistics-in-Fluid-Topics/Known-Limitations/Stemming-Dictionaries
doc.antidot.net
crawl=CC-MAIN-2021-25
8623
application/http; msgtype=response
sha1:M7FM7432SZECXRPYPGR5HAB6PAIYXX7E
<urn:uuid:c4c85188-0dc3-449e-a171-3e01c026a786>
2021-06-19T18:50:16Z
3.18.233.84
text/html
sha1:QHCABS6DLG6QBQYHFVX4QL32XST34RLG
<urn:uuid:b733404a-879b-4ca4-a949-e48508ea2198>
https://doc.antidot.net/r/3.2_public/Understand-Linguistics-in-Fluid-Topics/Known-Limitations/Stemming-Dictionaries
response
<urn:uuid:3f572ea1-68a1-4ebc-bc0d-e951fc29414d>
isPartOf: CC-MAIN-2021-25
publisher: Common Crawl
description: Wide crawl of the web for June 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-168.ec2.internal
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/