-
Notifications
You must be signed in to change notification settings - Fork 183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Readd Confluent Kafka drivers #418
base: master
Are you sure you want to change the base?
Conversation
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## master #418 +/- ##
==========================================
- Coverage 93.72% 91.62% -2.11%
==========================================
Files 102 103 +1
Lines 11154 11388 +234
Branches 1534 1564 +30
==========================================
- Hits 10454 10434 -20
- Misses 612 867 +255
+ Partials 88 87 -1
☔ View full report in Codecov by Sentry. |
Tested things with the [2022-12-22 11:30:58,684] [10809] [INFO] [^---Recovery]: Starting...
[2022-12-22 11:30:58,686] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:30:59,687] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:00,687] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:01,687] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:02,688] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:03,688] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:04,689] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:05,689] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:06,689] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:07,690] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:08,690] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:09,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:10,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:11,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:12,691] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
[2022-12-22 11:31:13,692] [10809] [INFO] [^---ConfluentConsumerThread]: Still waiting for assignment...
28535[2022-12-22 11:31:14,692] [10809] [ERROR] [^---Conductor]: Crashed reason=RuntimeError('This event loop is already running')
Traceback (most recent call last):
File "/home/redacted/PycharmProjects/faust/venv/lib/python3.8/site-packages/mode/services.py", line 843, in _execute_task
await task
File "/home/redacted/PycharmProjects/faust/faust/transport/conductor.py", line 293, in _subscriber
await self.app.consumer.subscribe(await self._update_indices())
File "/home/redacted/PycharmProjects/faust/faust/transport/consumer.py", line 1423, in subscribe
await self._thread.subscribe(topics=topics)
File "/home/redacted/PycharmProjects/faust/faust/transport/drivers/confluent.py", line 207, in subscribe
self._ensure_consumer().poll(timeout=1)
File "/home/redacted/PycharmProjects/faust/faust/transport/drivers/confluent.py", line 211, in _on_assign
self.thread_loop.run_until_complete(
File "/usr/lib/python3.8/asyncio/base_events.py", line 592, in run_until_complete
self._check_running()
File "/usr/lib/python3.8/asyncio/base_events.py", line 552, in _check_running
raise RuntimeError('This event loop is already running')
RuntimeError: This event loop is already running |
Keep an eye on confluentinc/confluent-kafka-python#1448, it would make more sense to use Confluent kafka in an asynchronous context. |
Don't know why they were removed in a0fb224, let's bring it back!