Skip to content

Conversation

@orolega
Copy link
Contributor

@orolega orolega commented Oct 29, 2025

Title

Fix IndexError in parallel request limiter v3 when handling multiple rate limit types per descriptor

Relevant issues

Fixes #15716

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • I have added a screenshot of my new test passing locally
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix
✅ Test

Changes

  • Fixed IndexError in async_pre_call_hook by replacing fragile index mapping (floor(i / 2)) with a robust lookup using descriptor_key in rate limit statuses.
  • Updated logic to match statuses to descriptors by key, supporting multiple rate limit types (requests, tokens, max_parallel_requests) per descriptor.
  • Added fallback handling for cases where a status's descriptor_key does not match any descriptor, ensuring graceful error messages.
  • Added/updated tests to cover scenarios with multiple rate limit types per descriptor and mismatched descriptor keys.
image

@vercel
Copy link

vercel bot commented Oct 29, 2025

@orolega is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: parallel_request_limit_v3 is buggy

1 participant