feat: Add comprehensive rate limit handling across API providers

[devin-ai-integration]

Add comprehensive rate limit handling across API providers

This PR implements robust rate limit handling across all API providers used in the AI-Scientist framework, addressing the continuous retry issue (SakanaAI#155).

Changes

• Add RateLimitHandler class for centralized rate limit management
• Implement provider-specific request queues and locks
• Add proper error handling and logging for rate limit events
• Extend backoff patterns to all API providers (OpenAI, Anthropic, Google, xAI)
• Add user feedback during rate limiting
• Add configurable minimum request intervals per provider

Implementation Details

• Created new rate_limit.py module for rate limit handling
• Added provider-specific rate limit detection
• Implemented request queuing mechanism
• Added comprehensive logging for debugging
• Extended backoff patterns with proper error type detection

Testing

The changes have been tested by:

• Verifying rate limit detection for different providers
• Testing backoff behavior with simulated rate limits
• Checking proper queue management
• Validating logging output

Impact

These changes make the system more robust by:

• Preventing continuous retries on rate limits
• Providing better error messages and logging
• Managing request rates across different providers
• Improving overall stability of API interactions

Fixes SakanaAI#155

Link to Devin run: https://app.devin.ai/sessions/2ec43d6fe7a84849a348753167e5a895

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR
• Look at CI failures and help fix them

⚙️ Control Options:

• Disable automatic comment and CI monitoring

Add "(aside)" to your comment to have me ignore it.