[Question]: When using vLLM to run Qwen3-32B, the model's performance has severely degraded.

[husl616]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Describe your problem

I'm using vLLM to run the Qwen3-32B model, but its performance has severely degraded. During conversations, the model fails to correctly return tokens like [ID:XX] in its responses, and its output quality is significantly worse than when using Ollama to run Qwen3-8B. Are there any specific parameters I need to configure when running Qwen3 with vLLM?

[dosubot]

There are no documented vLLM-specific configuration parameters in RAGFlow issues or docs that directly address Qwen3-32B's degraded performance or missing tokens like [ID:XX]. However, the codebase and community discussions highlight that degraded output and missing special tokens are often caused by tokenizer or prompt formatting mismatches, especially for Qwen3 models on vLLM.

For best results, double-check that your tokenizer configuration and prompt templates match Qwen3-32B's requirements—incorrect or missing special tokens can cause the model to skip or mangle output tokens. You may need to override the default tokenizer or prompt formatting in your vLLM deployment. Also, ensure you set context_window_size in vLLM to fit your GPU VRAM and concurrency needs, and that your model name and endpoint are correctly registered in RAGFlow's model provider settings.

If you recently updated RAGFlow, note that the Qwen3-32B config was changed to use max_tokens: 128000 and tags: LLM,CHAT,128k, which could affect context handling and output quality if your vLLM server expects different values (reference).

If you continue to see degraded output, focus on prompt and tokenizer alignment for Qwen3-32B with vLLM, as this is a common source of tokenization issues for large models.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[husl616]

Could you please advise what values should be set for frequency_penalty and presence_penalty when loading Qwen3-32B with vLLM?