-
Notifications
You must be signed in to change notification settings - Fork 6.6k
feat(vertex-vector-search): Add Google Vertex AI Vector Search v2.0 support #20351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
- Add api_version parameter for v1/v2 routing (default: v1) - Implement v2 operations using google-cloud-vectorsearch SDK - Add collection_id parameter for v2 collections - Add _v2_operations.py with all v2 CRUD operations - Add retry decorator with exponential backoff - Maintain 100% backward compatibility with v1 - Add 23 unit tests for v2 functionality - Add v2 demo notebook (VertexAIVectorSearchV2Demo.ipynb) - Add V2_MIGRATION.md migration guide - Update README with v2 examples - Add [v2] optional dependency for google-cloud-vectorsearch"
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
|
||
| ## Documentation | ||
|
|
||
| - **v1.0 Usage**: See the [v1 example notebook](../../../docs/examples/vector_stores/VertexAIVectorSearchDemo.ipynb) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would use https links, since this README also gets copy-pasted into a website webpage (which these links won't work)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a loooot of extra code here to support both v1 and v2? Is there a strong reason to do this vs. just doing a larger version bump and only supporting v2? How long is v1 being supported for?
|
|
||
| [project.optional-dependencies] | ||
| v2 = [ | ||
| "google-cloud-vectorsearch>=0.1.0", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a good reason to make this optional? Does it conflict with existing dependencies?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Side note, but I also just noticed vertex embeddings are a dependency, not sure why thats even there (especially since that package is deprecated in favor of google-genai embeddings)
Description
This PR adds support for Vertex AI Vector Search v2.0 (collection-based architecture) while maintaining 100% backward compatibility with v1.0 (index/endpoint architecture).
Summary of Changes
api_versionparameter (default:"v1") that routes operations to v1 or v2 implementations_v2_operations.pywith all CRUD operations usinggoogle-cloud-vectorsearchSDKindex_id/endpoint_id, v2 requirescollection_idVERTEX_AI_ENABLE_V2environment variable for safe rollbackDependencies
New optional dependency added:
google-cloud-vectorsearch>=0.1.0(only required for v2, installed viapip install 'llama-index-vector-stores-vertexaivectorsearch[v2]')Fixes #
New Package?
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?(This is not a new package - it's an update to the existing
llama-index-vector-stores-vertexaivectorsearchpackage. Thetool.llamahubsection already existed. We updatedpyproject.tomlto add the[v2]optional dependency.)Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)(Version remains at 0.3.2. Only added
[project.optional-dependencies]section for v2 SDK. Maintainers may want to bump version as part of merge.)Type of Change
How Has This Been Tested?
Test Details
23 new unit tests added for v2 functionality:
TestV2ParameterValidation(7 tests): Validates v2 requires collection_id, rejects v1 paramsTestV2Routing(5 tests): Verifies correct routing based on api_versionTestV2SDKImport(2 tests): Tests lazy loading of v2 SDKTestV2FeatureFlags(3 tests): Tests VERTEX_AI_ENABLE_V2 environment variableTestV2SDKManager(3 tests): Tests SDK manager v2 client initializationTestV2RetryDecorator(3 tests): Tests retry logic with exponential backoffAll existing v1 tests pass (backward compatibility verified)
V1 integration test passes
V2 integration test passes
V2 demo notebook tested (
VertexAIVectorSearchV2Demo.ipynb) - verified collection creation, document insertion, querying, and metadata filtering work end-to-endSuggested Checklist:
uv run make format; uv run make lintto appease the lint godsFiles Changed
base.py_v2_operations.py_sdk_manager.pypyproject.toml[v2]optional dependencytests/test_vector_stores_vertexaivectorsearch.pyREADME.mdV2_MIGRATION.mddocs/examples/vector_stores/VertexAIVectorSearchV2Demo.ipynbUsage Examples
Installation
V1 vs V2 Comparison
Related Documentation