Skip to content

Version 1.1
Compare
Choose a tag to compare
@Emily3403 Emily3403 released this 16 Feb 00:05
· 387 commits to main since this release
V1.1
28b9d53

isisdl version 1.1

A new major version is here! 🥳 🎉

Highlights:

  • isisdl-compress: Compress your videos into the H265 codec
  • Systemd timers
  • configuration file [linux exclusive]
  • New (experimental) installation method

Breaking changes

In order to account for the breaking changes I have introduced, the directory name has changed. Now it is called isisdl, located in your home directory. All courses are downloaded to there without the previous directory Courses/. Also, there is no .intern/ directory anymore, only the .state.db Database.

This should be the last time I rename this directory. Let's see if it will be that way!

Detailed Changelog

ffmpeg compression to H265

When executing isisdl-compress, ffmpeg is launched with options to compress your files (only for the courses you whitelist). These are sorted by maximum bitrate such that the files with the highest expected efficiency are compressed first. Thus, the efficiency will start out very high and drop gradually. It should be a smooth curve and the compression can be stopped at the point where you think compression is not worth it anymore. I set this point at 50%.

The compression is very efficient. I could compress the first 20 GiB into just 3 GiB. Of course over time this will drop down, but for the first few hours it is definitely worth it.

The compressed videos are also nicely integrated into isisdl. Compressed files are not re-downloaded if either of the following is true

  • You don't delete / lose the ~/isisdl/.state.db File
  • You don't rename compressed files

If you delete ~/isisdl/.state.db and rename the compressed files it still is not that big of a deal since they won't be overwritten (they have a different file name). The isisdl-sync prompt will, however, notify you of unrecognized files.

In version 1.2 I will use the mp4 metadata to deduce the original filename and this problem won't exist anymore.

Status

There is a detailed status info on

  • Total size before / after compression
  • Global efficiency
  • How much space was saved
  • How long the current file will take
  • Estimated current file size

Quitting while compressing

Sending a quit-signal to isisdl while in the middle of compressing is supported - the current compression will finish and an overview of the courses and their file-size-savings is displayed. However, if you quit isisdl while in the phase of waiting for the last compression to finish, ffmpeg will produce corrupted output. Since the output is saved to a temporary file first it is not a problem. Beware that you lose the progress that you have made for that file.

Side note: global time

It is not possible to calculate the global time since that would require starting every compression and then estimating the total time based on how much it took e.g. for 10s. However, if ffmpeg is interrupted during compression it can't be re-launched. And compression 10s of every video to throw it away again is horribly inefficient.

New (experimental) installation method

I have discovered the python module nuitka, which allows for compiling python code directly to native executables. These are huge: I am taking 20 MiB for a single one. This is due to the python runtime being insanely large. It is, however, acceptable since it provides great convenience. You don't have do install isisdl from pip but rather only download the executable(s) you need.

In theory this allows you to execute isisdl without having python installed. But don't quote me on that. I will test the compatibility and static-ness of the executables in the near future. For now it is an experimental installation method.

Unencrypted passwords are not stored unencrypted anymore

Previously if you selected to store the password unencrypted they were saved "as is" in the database. This in not really that problematic - since your computer should be free from malware - however it is better this way. This way of handling passwords is inspired from Google Chrome.

New configuration file

If you are running a Unix-like environment, you now have an example config located at ~/.config/isisdl/example.yaml once you execute isisdl for the first time. Here you can set every option that is available. You can do things like change the name of main working directory, database file name and other things you could set via the wizard.

In fact this file can also set options that you could not set with the wizard. All options which you set are interpreted by the settings.py file, which has all the static configuration. Here you could set things like the download_chunk_size, status_time or master_password to encrypt if no password was provided.

Usually you don't want to mess with these settings since they are chosen in the best way I could find. If you do find a setting that benefits from a different value do let me know.

Systemd-timers

Systemd-timers provide a huge feature set and are "better" than Cronjobs. See this and this link for further details. While I don't use the most part of the features it provides at least the dependency of cron is now eliminated. Of course, you will have to have a systemd-based distribution to take advantage of them. But who hasn't these days?

The location of the files is ~/.config/systemd/user/isisdl.timer while the service is located at ~/.config/systemd/user/isisdl.service. This timer is set to launch isisdl every hour, however it can be set to whatever time you feel is appropriate for your configuration.

Under the hood: config

I used to have a table in the Database responsible for configuration. A new value would be INSERT-ed into it. This way of doing it was syntactically very noisy and had a few own issues. Now there exists a class that takes care of it. Here attributes can be type-annotated and defined with __slots__ removing the functionality of arbitrarily setting attributes. A small caveat following from this is that json-strings are stored in the database. I've seen a meme about it, but I think this is the most convenient way to store it.

Also, the config values now have readable names instead of "0", "1", "2", …

Minor details

Better sanitation of filenames

Since the directory name changed the absolute path of every downloaded file now has no special characters in it. Also removed the option to only replace some characters - now it is all or nothing.

New update policy

Now isisdl has the ability to notify you about an update without actually installing it.

Better exception handling

All blocks that may fail are wrapped in a big

try:
    ...
except Exception:
    generate_error_message()

block. When an error occurs it is logged to a file and the program is exited gracefully.

Assets 10
Loading