transformer not recognizing this model
The checkpoint you are trying to load has model type lfm2 but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.
bump transformers version
did not work, what should I do
Hi! We just happen to be using the LFM2:0.3B as our model for training, and we noticed this thread, and figured we would share this: https://github.com/luna-system/ada-slm/blob/trunk/train_v9b_pure.py
ada-slm-v9b-pure is a fine-tune of LFM2 that teaches a glyphic language for machine understanding of logic and math! So it's a pretty basic implementation of fine-tuning this particular model!
So the above script is a reference implementation of an LFM2 retrain - complete with ROCm fixes, AND some VERY specific versions of Python and transformers that actually work well for this use case! Open source and public domain! Hope it helps <3
(Thank you LiquidAI team for the architecture! We're using it a LOT in our research!)