-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix writing large axml chunks #51
Conversation
cae274b
to
20ae287
Compare
Codecov ReportAttention:
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #51 +/- ##
==========================================
- Coverage 83.45% 83.23% -0.22%
==========================================
Files 6 6
Lines 677 698 +21
==========================================
+ Hits 565 581 +16
- Misses 112 117 +5 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great overall! Just a couple of comments
include/bw64/writer.hpp
Outdated
/// excpetions. If it does throw, this object may be in an invalid state, | ||
/// so do not try again without creating a new object. | ||
void close() { | ||
if (!closed_) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this flag be factored out by always closing the file even if errors occurred? we're considering any call to this a one-shot thing anyway with no retry. So, e.g, wrap the finalise calls in a try/catch, and close on catch with a pass-through of the exception? Can just use if(fileStream_.is_open()) then for the condition. Perhaps bad practice to "try" though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did briefly consider that, and went with this because i didn't really want to think about fstreams too much (e.g. can an error make is_open
return false?)
I think this is safe (supposedly is_open
is only cleared by close
), though maybe messier overall than one more flag. I've pushed a commit that changes this, what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I slightly prefer not having the flag, but mainly because the var name didn't really reflect the state. I.e, it's not necessarily closed if closed_ is true - that flags really just being use to prevent repeated attempts to close. So I guess I was more thinking the var name either needs to reflect what's it's doing or made to really track closed state (in which case is_open does that anyway).
I'm unsure whether not closing might cause hanging open files too? it probably gets properly closed in any case when the object is destructed tbh.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah good point, i think I'll stick with the exception based one.
The "closed" naming was just to match the "close" function, which i think makes sense especially for things working with files.
I'm unsure whether not closing might cause hanging open files too? it probably gets properly closed in any case when the object is destructed tbh.
I think it's fine, and works in testing, but I probably add a test of this in the normal set too (the big
ones don't get ran in CI).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at it again, it seems odd to have .close()
on writers but not readers, but it seems like a good opportunity to check for any file errors that haven't been spotted anyway, so i've added that. Sorry for the spam :)
this enables zero-copy construction if std::move is used it's unclear to me why vector was used in the first place -- sorry Chesterton
this can avoid a copy, and slowness of streams it would be nice to allow the data to be moved out of AxmlChunk, but Bw64Writer exposes a non-const pointer to its AxmlChunk, and can not handle a change in size
This is required when writing to properly catch exceptions raised during finalisation. The reader equivalent is added for symmetry.
90b9208
to
c70f78a
Compare
This required a few changes:
... and some other improvements along the way
Bw64Writer::close()
to be able to catch exceptions during finalisationAxmlData
, and accessing the string withAxmlData::data()
to avoid copies while reading/writingAxmlChunk
[big]
tests