fix: pdf conversion for long lines #17
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ublefo
force-pushed
the
pdf-long-lines
branch
2 times, most recently
from
27f120b to
c13c1f6
Compare
|
Can we update this PR so it is only changing files related to this fix. |
ublefo
force-pushed
the
pdf-long-lines
branch
3 times, most recently
from
f406f6c to
e776598
Compare
|
This is ready for review now |
PDF processing will fail with "! Dimension too large." if a line of text is way too long. Implement a simple processing helper method to call fold (from coreutils) to fold long lines for all code files.
We shouldn't modify student submissions, instead we use the temp file when rendering them to PDF. Cleanup will be performed after rendering is complete.
Run fold on provided files and compare the output with diff. If the file doesn't contain any lines that are over the configured threshold, it will be identical to the original. In this case we replace the temp file with a symlink for easy identification in the template.
Redirect stdout to /dev/null since we don't need the diff output
Set a hard limit of 1000 characters, and truncate everything in the same line after the limit is reached. Otherwise we could get PDF files with hundreds of pages which is completely unreadable.
|
Closed in favor of doubtfire-lms#439 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes the PDF processing issue when there are super long lines in the submitted code files by using
cutandfoldfrom coreutils to wrap lines. Additionally, introduce https://github.com/ruby/shellwords for proper shell escape, andpdf-readerfor extracting text from PDF files.There are two limits configured, the hard limit is 1000 characters, and the soft limit is 160 characters. Any lines over 1000 characters will be truncated, and then line breaks will be applied for any lines over 160 characters long.
The student submitted files are never modified, we write the processed results into temporary files, and the latex template will use those for rendering instead. The temp files are then cleaned up after rendering. If the rendered file has been modified, a warning will be added into the PDF document to indicate the rendered file differs from the original submission.
A unit test and a test file have been added to test this specific feature.
Supersedes #8
Generated result:
