Skip to content
This repository has been archived by the owner on Mar 24, 2021. It is now read-only.

[WIP] Support RecordBatch #844

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Conversation

emmettbutler
Copy link
Contributor

@emmettbutler emmettbutler commented Jul 26, 2018

This pull request addresses #831 by adding support for RecordBatch message formatting to pykafka. This includes supporting the encoding and decoding of varints via the struct_helpers interface.

@codecov-io
Copy link

codecov-io commented Jul 26, 2018

Codecov Report

❗ No coverage uploaded for pull request base (master@8747398). Click here to learn what that means.
The diff coverage is 75.71%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #844   +/-   ##
=========================================
  Coverage          ?   82.38%           
=========================================
  Files             ?       36           
  Lines             ?     3850           
  Branches          ?      529           
=========================================
  Hits              ?     3172           
  Misses            ?      526           
  Partials          ?      152
Impacted Files Coverage Δ
pykafka/protocol/message.py 79.87% <21.05%> (ø)
pykafka/utils/struct_helpers.py 97.95% <96.07%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8747398...7370c9d. Read the comment docs.

@emmettbutler
Copy link
Contributor Author

The protocol logic for RecordBatch is implemented and untested. The next step is to figure out how to get Kafka to start speaking in RecordBatch instead of MessageSet so I can run some tests to work out the protocol implementation bugs. Once that's done, the next step will be to make changes to the producer and consumer to have them use RecordBatch when appropriate.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants