-
Notifications
You must be signed in to change notification settings - Fork 8.3k
USearch HNSW index: code: 1001 Magic header mismatch - the file isn't an index #53767
Description
Describe what's wrong
I attempted to make a table with the usearch index and query it via the cosineDistance function. This is as specified in the docs on earch indexes which were merged in #53447 a couple days ago.
Querying the index only resulted in std::exception. Code: 1001, type: std::runtime_error, e.what() = Magic header mismatch - the file isn't an index (version 23.8.1.2501 (official build)) rather than resulting rows.
I am working on this as a part of an effort to switch off of Qdrant for arguflow.ai
The easiest way to reproduce this is to use the jupyter notebook on this by arguflow.ai - the README and first cell of the notebook contain steps to repro.
Does it reproduce on recent release?
Yes, 23.8.1.2501
I enabled crash reporting, so hopefully a Sentry report went through when this failed.
How to reproduce
- Use
23.8.1.2501 - Make a table with a
usearchindex likeINDEX usearch_embedding_vector embedding_vector TYPE usearch('cosineDistance') GRANULARITY 100 - Attempt to query the table with
cosineDistancelikeSELECT point_id, cosineDistance(embedding_vector, {first_vector}) AS score FROM qdrant_benchmark_10 ORDER BY score ASC LIMIT 10 - Sample data is in the jupyter notebook on this by arguflow.ai
Expected behavior
Rows should get returned
Error message and/or stacktrace
std::exception. Code: 1001, type: std::runtime_error, e.what() = Magic header mismatch - the file isn't an index (version 23.8.1.2501 (official build))
Additional context
I am hopeful that usearch is more accurate than annoy . I'd be happy to contribute if someone adds a rough outline on what it would take to fix this. Fwiw, the same error occurs with L2distance as shown in this notebook.