DAN™
|
889bdd7686
|
command-r : add BPE pre-tokenization (#7063)
* Add BPE pre-tokenization for Command-R/R+.
* Bump transformers convert requirement.
* command-r : add individual digits regex
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
|
2024-05-05 08:19:30 +03:00 |
|