-
Notifications
You must be signed in to change notification settings - Fork 668
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error: xception raised in Job[0]: APIConnectionError(Connection error.) when split one big batch into several small batches to submit #1423
Comments
Hey there, @francescofan! I'm here to help you with any bugs, questions, or contributions you have. Let's squash that bug together! The
By adjusting these configurations and checking the network conditions, you should be able to reduce the occurrence of these errors. |
@dosu |
To enable more detailed logging in your Ragas setup, you can set the environment variable export __RAGAS_DEBUG_TRACKING=True This should provide more detailed logs that can help diagnose the issues you're facing with |
@dosu I am in a windows environment. So how to set up export __RAGAS_DEBUG_TRACKING=True |
In a Windows environment, you can set the environment variable using the set __RAGAS_DEBUG_TRACKING=True Run this command in the Command Prompt before executing your Python script. This will enable detailed logging for your current session. If you need this setting to persist across sessions, you can add it to your system environment variables through the System Properties. |
@dosu |
I haven't encountered that specific issue directly, but it sounds like it could be related to how the Ragas framework handles API connections or server-side rate limiting. Since you've already tried various methods to address the issue, consider checking if there are any server-side restrictions or rate limits that might be causing the second request to fail. Additionally, ensure that your network connection is stable and that there are no intermittent issues affecting the connection. If the problem persists, enabling detailed logging as previously discussed might help identify any underlying issues. |
[ ] I have checked the documentation and related resources and couldn't resolve my bug.
Describe the bug
I set up a defination to split the input Dataset into several batches to submit. For example, when a big batch which includes 200 questions/answers/ground_truth/contexts content. I will figure it out and split it into 20 batches which contains 10 questions per batch. And when the result of first batch is returned and next batch is submitted there will be an error like the following.
xception raised in Job[0]: APIConnectionError(Connection error.)
Exception raised in Job[3]: TimeoutError()
Exception raised in Job[2]: TimeoutError()
Exception raised in Job[1]: TimeoutError()
I don't know why this camed.
I suppost it is just like a loop submit.
Ragas version: 0.1.20
Python version: 3.11.9
LLM: Deepseek v2.0
Embedding: customized embedding : embeddings = MyCustomEmbeddings("http://customizedEmbbding something like this")
Code to Reproduce
Error trace
Evaluating: 0%| | 0/4 [00:00<?, ?it/s]Exception raised in Job[0]: APIConnectionError(Connection error.)
Exception raised in Job[3]: TimeoutError()
Exception raised in Job[2]: TimeoutError()
Exception raised in Job[1]: TimeoutError()
Evaluating: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [03:00<00:00, 45.00s/it]
All metrics is nan: 'faithfulness': nan, 'context_recall': nan, 'context_precision': nan, 'answer_correctness': nan
Expected behavior
It will not occur APIConnectionError(Connection error.) and finish succesfully.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: