GermanTokenizerPostProcessor (Stanford CoreNLP API)

java.lang.Object
- edu.stanford.nlp.process.CoreLabelProcessor
- - edu.stanford.nlp.international.german.process.GermanTokenizerPostProcessor

```
public class GermanTokenizerPostProcessor
extends CoreLabelProcessor
```
German tokenization is handled by PTBLexer. This module handles special cases for German. Currently the German tokenization in Stanford CoreNLP should match the UD standard, as seen in the CoNLL-2018 dependency parsing data for example. Two special cases for now: - rebuild ordinals - rebuild numeric ranges For instance in the sentence "Der Vertrag läuft offiziell bis zum 31. Dezember 1992." the "31." should be a token. This module will merge "31" and "." back together. Also "1989" "-" "1990" should be merged back to "1989-1990"

Fields
Modifier and Type	Field and Description
`java.util.function.Function<CoreLabel,java.lang.Boolean>`	`afterIsEmpty` Check that after() is not null and the empty string
`java.util.HashSet<java.lang.String>`	`germanAbbreviations`
`java.util.HashSet<java.lang.String>`	`ordinalPredictingWords`

Constructors
Constructor and Description
`GermanTokenizerPostProcessor()`

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static void`	`mergeTokens(CoreLabel token, CoreLabel nextToken)` merge the contents of two tokens
`java.util.List<CoreLabel>`	`process(java.util.List<CoreLabel> tokens)` Alter the tokenization of list of tokens (e.g.
`java.util.List<CoreLabel>`	`restore(java.util.List<CoreLabel> originalTokens, java.util.List<CoreLabel> processedTokens)` Undo the tokenization changes of process, maintaining any tagging

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

public java.util.function.Function<CoreLabel,java.lang.Boolean> afterIsEmpty

Check that after() is not null and the empty string

public java.util.HashSet<java.lang.String> ordinalPredictingWords

public java.util.HashSet<java.lang.String> germanAbbreviations

Constructor Detail
- GermanTokenizerPostProcessor
```
public GermanTokenizerPostProcessor()
```

Class GermanTokenizerPostProcessor