bigscience

/

bloom

like 5k

Follow

BigScience Workshop 1.31k

The model hasn't been pre-trained on Korean so performance will be worse, but it does still know a bit about Korean, so you can definitely use Korean in your prompts 😇

Can I contribute to this project for the Bloom model to support the Korean language?
If so, how can I help?

Yeah I think there are some efforts on teaching the model new languages. The people from this paper https://arxiv.org/abs/2204.04873 are probably good to reach out to.
Tagging @yongzx here 🤗

@MasBakr Yes there's currently efforts in adapting models to new languages, and currently we are focusing on low-resource languages to adapt to. Are you in the BigScience community by any chance?

@MasBakr I'm currently trying to support Korean and others right now. Have you had any success?

@MasBakr I'm currently trying to support Korean and others right now. Have you had any success?

You can find the results of the research on adding a new language (actually we also tried Korean) & open-source code via this link: https://arxiv.org/abs/2212.09535
@yongzx lead this research - He may have more insights for you!

@MasBakr @kmdanikowski

We actually tried adapting to Korean (still uploading the models to https://huggingface.co/bs-la) and evaluating on XNLI and PAWS-X. Happy to discuss if there's any lingering question / future work you plan to explore.