diff --git a/98_pure_python_eventlog_parsing.md b/98_pure_python_eventlog_parsing.md index 4f34beb..bdaa636 100644 --- a/98_pure_python_eventlog_parsing.md +++ b/98_pure_python_eventlog_parsing.md @@ -1,39 +1,4 @@ - -# enhancement-NNNN: Your short, descriptive title - - - - +# Python Eventlog Parsing for Measured Boot Attestation - [Release Signoff Checklist](#release-signoff-checklist) @@ -57,193 +22,238 @@ template. ## Release Signoff Checklist - - - [ ] Enhancement issue in release milestone, which links to pull request in [keylime/enhancements] - [ ] Core members have approved the issue with the label `implementable` - [ ] Design details are appropriately documented - [ ] Test plan is in place - [ ] User-facing documentation has been created in [keylime/keylime-docs] -## Summary +## Proposal summary The proposal is to replace the dependency on `tpm2-tools` in the Keylime measured boot attestation code with native python -code. Motivation in detail just below. At the simplest level the -change is just to change the call to the event log parser from -`os.system` to native Python. However, additional complexity reduction -can be achieved by removing the Python based post-processing of the -event log and merging that code into the parser. - -We propose to keep the event log parser as a separate project within -the Keylime code base, and to package it independently. This was -proposed by the project maintainers as a way to keep disruptions in -Keylime to a minimum; once the python based event log parser is +code. + +Most of the change in Keylime itself to the nature of the parser's +invocation -- from `os.system` to native Python. Additional complexity +reduction can be achieved by removing the Python based post-processing +of the event log and merging that code into the parser. Current +features of post-processing include correct handling of UUIDs (badly +parsed in earlier versions of `tpm2-tools`) and `libefivar` +enhancements to parse device path entries. + +We propose to keep the event log parser itself as a separate project +within the Keylime code base, and to package it independently. This +was proposed by the project maintainers as a way to keep disruptions +in Keylime to a minimum; once the python based event log parser is packaged, it can be listed as a dependency on Keylime itself. To this end, the python event log parser will have its own CI and unit tests, code checks, and its own packaging mechanisms. - ## Motivation -The chief motivation for this proposal is that keeping Keylime measured -boot attestation alive and functional over the last two years has -resulted in at least two incidents in which upgrading `tpm2-tools` to -a new version caused measured boot attestation to malfunction, because -the output format (nominally YAML) of the tool was changed. - -The changes in `tpm2-tools` are typically benign when viewed in context. Some -examples: trailing zeroes were excluded, now being excluded; multiline -output handled incorrectly; certain input parsed to strings of -incorrect length; UUIDs being interpreted incorrectly as in network -order, then as little endian. - -However, the *number* of such changes, and the ongoing frequency of -such changes, makes the maintenance of Keylime challenging. - -Basically, we propose to replace the approx. 1000 lines of C code in the -tpm2-tools package with a mostly equivalent length (700 lines at -present) of native Python code, maintained by the Keylime project. - +The chief motivation for this proposal is to keep measured boot +attestation safe from bit rot. We have, in the last two years of +operating Keylime (2021 to 2023), experienced at least two incidents +in which upgrading `tpm2-tools` to a new version caused measured boot +attestation to malfunction, because the output format (nominally YAML) +of the tool was changed. + +Generally speaking the quality of `tpm2-tools` has been improving. +Some examples of disruptive changes we have encountered: +* Trailing zeroes used to be excluded, now are excluded; +* multiline output handled incorrectly in `tpm2-tools` v. 5.5 +* certain input parsed to strings of incorrect length, resulting in utf-16 garbage +* UUIDs being interpreted incorrectly as in network order in v. 5.11 + +These are generally easy to correct for, but require us to actively +follow changes in the tooling, and each new version of `tpm2-tools` +requires us to adjust. + +Therefore we propose to replace the approx. 1000 lines of C code in +the `tpm2-tools` package with a mostly equivalent length (700 lines at +present) of native Python code, maintained by the Keylime project. The +code savings come from the fact that OO features of Python allow for +simpler, more natural code organization; Python `struct` is very +effective at parsing binary structures into Python native objects; and +lastly, Python has JSON support. ### Goals -Goal 1: keylime project -Open questions: - -Goal 2: a PR in keylime to deploy python event log parser - +* Improve the stability of measured boot attestation over time. +* Improve the quality of the measured boot event log parser. +* Reduce Keylime's dependency on foreign tooling. ### Non-Goals An explicit non-goal of _this_ proposal is to also enhance the -tpm2-tools package by enumerating the issues found during the +`tpm2-tools` package by enumerating the issues found during the rewrite. That (entirely reasonable) work item should be handled somewhere else. We do not (in this proposal) aim to change anything else about -measured boot attestation in Keylime. To that end, the python event -log parser is slated to produce compatible output with what -`tpm2-tools` is producing right now. +measured boot attestation in Keylime. The python event +log parser is designed to produce output comparible with +`tpm2-tools`. -## Proposal +### Notes/Constraints/Caveats (optional) - +The most obvious caveat to address is that this project is basically +duplicating code that already exists in `tpm2-tools`, and therefore +could be seen as a duplication of effort, unwise both because of the +wasted labor and the implied long tail of required maintenance. -### User Stories (optional) +The developers' argument is that (a) the duplication of effort is not +that great, since it was done in 2 weeks of part-time work, minus the +organizational aspects such as writing this document (b) the long tail +of management is not really that long, because the specifications we +are following are changing very slowly. - +In any case we think that the implicit advantages of python-based +development (e.g. native JSON support) and the much shorter +development loop (i.e. the event log parser's developers and the end +users of Keylime overlap) mitigate the disadvantage of code +duplication. -#### Story 1 +### Risks and Mitigations -#### Story 2 +The largest risk is that the Python based event loop stops being +maintained, develops bit rot and contributes to a decline of the +quality of the Keylime code base. Mitigation lays in the decision to +continue to use JSON as the exchange format between the parser and the +Keylime measured boot policy agent, which would make a move *back* to +`tpm2-tools` possible at least in principle. + +There are no serious security aspects regarding the event log +parser. To the extent that the measured boot event log is +self-checking (i.e. event digests can be recalculated for certain +types of events), the event log parser is already doing so. NB we are +not aware of the `tpm2-tools` based event log parser doing any +self-checking. And of course the python based event log parser is +capable of generating PCR reference values in the same way +as `tpm2-tools`. + +Corrupt or maliciously crafted binary event logs have the capability +to disrupt the parser (insufficient self checks have been implemented +to date). To the best of our knowledge such events merely result in +the parsing being aborted with an python Exception. -### Notes/Constraints/Caveats (optional) +## Design Details -To fill in: an extra project, extra code, possible divergence from `tpm2-tools`, duplicating effort. +The parser is implemented as a single Python file, `eventlog.py`. It +includes an `EventLog` class, which is `list` of `Event` objects. -### Risks and Mitigations +An `EventLog` object can be instantiated with a binary buffer +containing the binary event log, and becomes a list of the individual +events in the log. - +Currently only three operations are implemented on the EventLog class +itself: (a) parse into JSON, (b) self-verification and (c) generate a +list of PCRs for authentication against a TPM quote. -## Design Details +Typical use case for the EventLog class: - +``` + args = parser.parse_args() + assert args.file, "file argument is required" -### Test Plan + with open (args.file, 'rb') as fp: + buffer = fp.read() + evlog = eventlog.EventLog(buffer, len(buffer)) + print(json.dumps(evlog, default=lambda o: o.toJson(), indent=4)) +``` - +The event log parser has its own test suite (10 different event logs +collected from real machines). The comparison is performed against +`tpm2_eventlog` from `tpm2-tools` version 5.5. + +Attached is example output from the CI system: + +``` ++------------------------------+-------+-------+-------+---- +| log file | #Evts | #Fail | Pct. | msg. ++------------------------------+-------+-------+-------+---- +|bootlog-5.0.0-rhel-20210423T13| 34| 0| 0.00%| +|thyme-eventlog.bin | 39| 0| 0.00%| +|p511.bin | 58| 0| 0.00%| +|swtpm_measurements.bin | 104| 0| 0.00%| +|puiterwijk.bin | 101| 0| 0.00%| +|201123-eventlog.bin | 60| 0| 0.00%| +|ideapad1.bin | 39| 0| 0.00%| +|inspur.bin | 101| 0| 0.00%| +|rk049-s26.bin | 47| 0| 0.00%| +|intel_svr.bin | 119| 0| 0.00%| +|css-flex14vm4-bootlog.bin | 106| 1| 0.94%| ++------------------------------+-------+--------+------+ +| Totals: | 808| 1| 0.12%| ++------------------------------+-------+--------+------+ +``` + +TODO: integration testing +TODO: antagonistic testing (broken event logs) +TODO: testing of PCR generation +TODO: self-check testing +TODO: coverage testing ### Upgrade / Downgrade Strategy - +TODO -### Dependencie requirements +### Dependency requirements - +* TRANSFER: transferring the current Python-native event log parser into the +Keylime project. The current implementation +(https://github.com/galmasi/python3-uefi-eventlog) has its own CI +tooling based on Github Actions, which includes pylint, python type +checking and extensive functional testing against `tpm2_eventlog` by +comparing JSON outputs for a collection of 10 different event logs. -## Drawbacks +* TESTING: The developer[s] would absolutely welcome contributions of +other binary event logs for even more extensive functional +testing. Including maliciously crafted binary event logs designed to +trip up the parser. - +* PACKAGING: There is no packaging code implemented in the project at +this point. The developer[s] would welcome both help with the `pypy` +packaging code and with Red Hat/Canonical packages. -## Alternatives +* PEER REVIEW: The code was written mostly by a single person. A peer +review of the code would be welcome. - +* DOCUMENTATION: A lot of the event log parser is based on TCG +documentation. Better documentation (attributing parts of the code to +specific TCG and EFI documentation, by chapter and verse) would be +welcome. -## Infrastructure Needed (optional) +* INTEGRATION: a PR in keylime to replace the `tpm2-tools` call with +the native call. Goal 1 would be a pre-requisitve, since the +implication would be that the python event log parser is a package +dependency of keylime. -