Goes off the rails
#22
by Hypersniper - opened
Not a bad release! Needs some more training for sure. After a minute or so of conversation it starts to degrade in quality. Maybe more turn based conversations are needed for the dataset?
Could u share your inference script
Right. Its focus was to showcase naturalness, and basic instruction following, voice prompting. And it has a 2048 token window and only on minimal SFT data. Future versions will have proper post-training/alignment.
royrajarshi changed discussion status to closed