Skip to content

Conversation

@Akhil-Pathivada
Copy link
Contributor

@Akhil-Pathivada Akhil-Pathivada commented Nov 7, 2025

Problem

OpenSearch 3.x+ enables knn.derived_source.enabled by default, which allows OpenSearch to derive vectors from the vector values file instead of storing them in the _source field. However, users need a way to control this behavior for their specific use cases:

  • When working with different OpenSearch versions because this feature was not be available (< 3.x)
  • When they want to disable this feature to store vectors in _source for specific requirements

Solution

Added a new configurable option knn_derived_source_enabled allowing users to:

  • "None" (default) - Skip setting for OpenSearch < 3.x compatibility
  • "True" - Explicitly enable (3.x+ default behavior)
  • "False" - Disable for better performance on 3.x+

Changes

  • Added knn_derived_source_enabled: str | None = None field to OSSOpenSearchIndexConfig
  • Conditional setting in _create_index() - only added when explicitly set
  • UI dropdown with 3 options and version guidance
  • Added enum to CaseConfigParamType

Benefits

  • ✅ Version-compatible (OpenSearch < 3.x and 3.x+)
  • ✅ User-friendly UI with clear guidance
  • ✅ No breaking changes - default behavior unchanged

Files Changed

  • vectordb_bench/backend/clients/oss_opensearch/config.py
  • vectordb_bench/models.py
  • vectordb_bench/backend/clients/oss_opensearch/oss_opensearch.py
  • vectordb_bench/frontend/config/dbCaseConfigs.py

Screenshots

Screenshot 2025-11-07 at 17 04 59

@sre-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Akhil-Pathivada
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Akhil-Pathivada Akhil-Pathivada force-pushed the feat/opensearch-knn-derived-source branch from 7327025 to 0cd4a11 Compare November 7, 2025 12:39
@Akhil-Pathivada Akhil-Pathivada marked this pull request as ready for review November 7, 2025 12:57
@Akhil-Pathivada
Copy link
Contributor Author

/assign @alwayslove2013

@Akhil-Pathivada
Copy link
Contributor Author

Hi @alwayslove2013 I've opened a foundational PR #635, which introduces a new version-aware system for OpenSearch settings.

I'm converting this PR to a draft as it depends on the foundational system in #635. Once that merges, I'll update this branch with the new design and mark it as ready for review.

@Akhil-Pathivada Akhil-Pathivada marked this pull request as draft November 8, 2025 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants