Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error after building the graph #27

Open
MNLubov opened this issue Sep 20, 2022 · 2 comments
Open

Error after building the graph #27

MNLubov opened this issue Sep 20, 2022 · 2 comments

Comments

@MNLubov
Copy link
Contributor

MNLubov commented Sep 20, 2022

I am trying to use HNSW for the search and have the following problem.
After building the graph, I got the folloiwng error:
ArgumentError: invalid index: -0.017805025f0 of type Float32
I got this error here:
@inline distance(hnsw, i, j) = @inbounds evaluate(hnsw.metric, hnsw.data[i], hnsw.data[j])
How could this happen, if graph is build without problems?

@JonasIsensee
Copy link
Member

Hi @MNLubov,
can you give me a code example?

@MNLubov
Copy link
Contributor Author

MNLubov commented Sep 22, 2022

@JonasIsensee
I found what the problem is following. In HNSW distance for knn-search are calculated only for vectors, while for the graph building type of the data is not specified.

@inline distance(hnsw, i, j) = @inbounds evaluate(hnsw.metric, hnsw.data[i], hnsw.data[j])
@inline distance(hnsw, i, q::AbstractVector) = @inbounds evaluate(hnsw.metric, hnsw.data[i], q)
@inline distance(hnsw, q::AbstractVector, j) = @inbounds evaluate(hnsw.metric, hnsw.data[j], q)

I used distance for matrices, not vectors. My distance measure is based on the product of two matrices.
Is it possible to extend allowed types for the distance calculation in HNSW? I personally need Float32 matrices:

@inline distance(hnsw, q::Matrix{Float32}, j) = @inbounds evaluate(hnsw.metric, q, hnsw.data[j])
@inline distance(hnsw, i, q::Matrix{Float32}) = @inbounds evaluate(hnsw.metric, q, hnsw.data[i])

Also such addition leads to a change for length assertion:

@assert length(q)==length(hnsw.data[1])

I strongly rely on your package and it would be wonderful if it could be used
for any type of data with scalar distance. Moreover change of data type and extension of possible scalar distances
doesn't change logic of hnsw search.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants