[Question]: when i query, whey always the same contents being sent to LLM

[axz91]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

when i query, whey always the same contents being sent to LLM

[dosubot[bot]]

🤖

Hello,

The issue you're experiencing could be due to the way the query method in the DynamoDBVectorStore class is implemented. This method retrieves all items from the embedding collection and filters them based on the node IDs provided in the query. If no node IDs are provided, all embeddings are returned. The method then calculates the top K embeddings that are most similar to the query embedding and returns them. If the same embeddings are always being returned, it could be because the query embedding is always the same, or because the embeddings in the DynamoDB table are not diverse enough.

Here is the relevant code:

def query(self, query: VectorStoreQuery, **kwargs: Any) -> VectorStoreQueryResult:
    """Get nodes for response"""
    if query.filters is not None:
        raise ValueError(
            "Metadata filters not implemented for SimpleVectorStore yet."
        )

    items = self._kvstore.get_all(collection=self._collection_embedding).items()

    if query.node_ids:
        available_ids = set(query.node_ids)
        node_ids = [k for k, _ in items if k in available_ids]
        embeddings = [v[self._key_value] for k, v in items if k in available_ids]
    else:
        node_ids = [k for k, _ in items]
        embeddings = [v[self._key_value] for k, v in items]

    query_embedding = cast(List[float], query.query_embedding)
    if query.mode in LEARNER_MODES:
        top_similarities, top_ids = get_top_k_embeddings_learner(
            query_embedding=query_embedding,
            embeddings=embeddings,
            similarity_top_k=query.similarity_top_k,
            embedding_ids=node_ids,
        )
    elif query.mode == VectorStoreQueryMode.DEFAULT:
        top_similarities, top_ids = get_top_k_embeddings(
            query_embedding=query_embedding,
            embeddings=embeddings,
            similarity_top_k=query.similarity_top_k,
            embedding_ids=node_ids,
        )
    else:
        raise ValueError(f"Invalid query mode: {query.mode}")

    return VectorStoreQueryResult(similarities=top_similarities, ids=top_ids)

To resolve this issue, you might need to check how your query embeddings are being generated and ensure they are unique for each query. Also, check how your DynamoDB table and query engine are set up, and ensure they are properly configured to handle different queries and return different results.

You can find more details in the source code of the LlamaIndex repository:

• llama_index/tools/tool_spec/load_and_search/base.py
• llama_index/chat_engine/condense_question.py
• llama_index/vector_stores/dynamodb.py

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Best,
Dosu

Sources

• llama_index/tools/tool_spec/load_and_search/base.py
• llama_index/chat_engine/condense_question.py
• llama_index/vector_stores/dynamodb.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[axz91]

now sure what i need to do

[EmanuelCampos]

@axz91 can you share a more reproducible example? which models, embeddings, vector db