StreamBuffer for input #351
Conversation
MikeIndiaAlpha
requested review from
thomshutt,
emranemran,
AlexKordic and
cyberj0g
| void buffer_put_bytes(StreamBuffer *buffer, uint8_t *bytes, int64_t size); | ||
| void buffer_end_of_stream(StreamBuffer *buffer); | ||
| void buffer_error(StreamBuffer *buffer, StreamErrors error); |
There was a problem hiding this comment.
Consider adding unit test
There was a problem hiding this comment.
Yeah, right now these are tested with every test but pipe ones, but in future I guess I have to choose something.
| } | ||
| } | ||
|
|
||
| static int buffer_read_function(void *user_data, uint8_t *buf, int buf_size) |
There was a problem hiding this comment.
Consider adding unit test
:-)
I first want to change it into thread (goroutine?) to have synchronisation test. Then it could be removed, or alternatively it could be used to run a few tests of the demuxer kind with the StreamBuffer input?
The version that we have right now runs all tests that aren't using pipes through loadInputBuffer. I don't think it should be left this way, more like development help (having various tests and demuxers allowed me to understood requirements of the demuxer wrt to |
MikeIndiaAlpha
force-pushed
the
ma_stream_buffer
branch
3 times, most recently
from
a7f4e40
to
b53703a
Compare
MikeIndiaAlpha
force-pushed
the
ma_refactoring_second
branch
from
9196c7c
to
cea0459
Compare
MikeIndiaAlpha
force-pushed
the
ma_stream_buffer
branch
from
b53703a
to
c7ae831
Compare
MikeIndiaAlpha
force-pushed
the
ma_refactoring_second
branch
from
cea0459
to
c712137
Compare
MikeIndiaAlpha
force-pushed
the
ma_stream_buffer
branch
2 times, most recently
from
d3748e2
to
13446f1
Compare
Alternative input system for LPMS. The idea is to allow using of FFmpeg demultiplexers with any input mechanism, such as network packet delivery. In order to do that we have to provide custom AVIOContext with custom read() and seek() operations. This also means that synchronization is needed because we will have to perform input and transcoding in different threads (for example demux - which is a part of transcode - may want more data which is not available yet, then transcode should block until input thread delivers more data)
MikeIndiaAlpha
force-pushed
the
ma_stream_buffer
branch
5 times, most recently
from
13446f1
to
c76aec4
Compare
|
@AlexKordic - this is current version of the LL input/output. It works by having circular buffer on the input side (like we discussed) and packet queue on the output side. ffmpeg.go was modified to use LL input/output with all the tests. Namely, input file is loaded into stream buffer before First commit works with all the tests. Alas, second commit fails some. This is primarily because Apart from that, TestTranscoder_API_AlternatingTimestamps test fails for some reason not entirely clear for me (yet). From what I managed to see, .md5 muxer fails to send any data to custom I/O. But if you replace go procedures loadInputBuffer/storeOutputQueue with threads receiving/sending the data over the network, Transcoder should work with LL. Alternatively, when I'll be back I want to replace these procedures with file I/O threads myself, to make sure that synchronisation is correct. |
Output side counterpart to the StreamBuffer. It is not exactly the same solution, because input and output characteristics are quite different (for one, on the input side stream is just a series of bytes, whereas on the output side we are able to tell packets apart, assign timestamps, packet types, etc). There is simple Golang code for writing the data into the files, so that some tests will pass with this code. Note that the big problem with the queue output is that some muxers (for example mp4) need to be able to seek() in output, and we don't allow that.
MikeIndiaAlpha
force-pushed
the
ma_stream_buffer
branch
from
c76aec4
to
c74c5c6
Compare
Alternative input system for LPMS. The idea is to allow using of
FFmpeg demultiplexers with any input mechanism, such as network
packet delivery. In order to do that we have to provide custom
AVIOContext with custom read() and seek() operations. This also
means that synchronization is needed because we will have to
perform input and transcoding in different threads (for example
demux - which is a part of transcode - may want more data which
is not available yet, then transcode should block until input
thread delivers more data)