Modifying Llama 3’s tokenizer to tokenize numbers from right to left (R2L) instead of left-to-right (L2R) with just a few lines of code 🧑💻. This affect the grouping of numbers by three:
Modifying Llama 3’s tokenizer to tokenize numbers from right to left (R2L) instead of left-to-right (L2R) with just a few lines of code 🧑💻. This affect the grouping of numbers by three: