PaddleFormers 0.3 is officially released! This release introduces several key features and improvements:
✨ New Features
1. Hugging Face safetensor weight loading & saving
PaddleFormers now supports loading and saving Hugging Face safetensor model weights.
from paddleformers.transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-0.6B-Base",
convert_from_hf=True
)
model.save_pretrained("Qwen/Qwen3-0.6B-Base-new", save_to_hf=True)2. New model support
Added support for the following models:
- qwen2
- qwen3
- qwen2moe
- qwen3moe
- ernie4_5
- ernie4_5_moe
- gpt_oss
3. Generalized large model modules (paddleformers/nn)
Introduced a generalized module library for large models to reduce the cost of integrating distributed training.
Includes:
- Attention
- Embedding
- Pipeline parallel model
- Normalization
- MLP
- LM Head
- Linear
You can check out the implementation details here:
https://github.com/PaddlePaddle/PaddleFormers/tree/develop/paddleformers/nn