sentence1: we use a shared subword vocabulary by applying byte-pair encoding to the data for all variants concatenated .
sentence2: following koehn and knowles , we process all the data with byte-pair encoding to construct a vocabulary of 50k subwords .