Skip to content

Performance improvements, compressed I/O, and variable-name escaping

Compare
Choose a tag to compare
@johnkerl johnkerl released this 29 Dec 20:40
  • RFC-CSV read performance is dramatically improved and is now on par with other formats; read performance for all formats is slightly improved as well.
  • Variable names can now be escaped, using curly braces if there are special characters in the input-data field names. Example: mlr put '${bytes.total} = ${bytes.in} + ${bytes.out}'. See also #77 where this was requested.
  • Compressed I/O is now supported, using built-in compatibility with local system tools: http://johnkerl.org/miller/doc/reference.html#Compression. See also #77 where this was requested.
  • mlr uniq is now streaming (bounded memory use, functionality in tail -f contexts) when possible: i.e. when -n and -c are not specified.
  • Thorough valgrind-driven testing has been used to tighten memory usage. This is mostly an invisible internal improvement, although it has a slight across-the-board performance improvement as well as allowing Miller to handle even larger files in limited-memory contexts.