Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: pkcs11-tool based HSM support for factory #278

Merged
merged 17 commits into from
Oct 5, 2023
Merged

Commits on Oct 3, 2023

  1. Feature: Enable Golang implementation of PKI globally

    We have already tested this implementation on Windows in the field.
    Also tested it on Linux using a local build.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    2126abe View commit details
    Browse the repository at this point in the history
  2. Bugfix: Golang PKI impl should set IsCA=False for TLS cert

    This small discrepancy was found by the newly added unit tests.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    f57a408 View commit details
    Browse the repository at this point in the history
  3. Bugfix: Golang PKI would panic on malformed CSR or Cert PEM data

    Although this is not critical for the CSR which is warranted to be properly formatted by our API,
    it is really a problem for the certificate file loaded from the user filesystem.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    92c395b View commit details
    Browse the repository at this point in the history
  4. Cleanup: extract Bash based implementation from ota-lite into fioctl

    This might seem contradicting, but allows for some benefits:
    - No shell script files need to be created; they are piped into the Bash in-memory.
    - All temporary csr files are created/removed in /tmp; only necessary files stay in PKI directory.
    - Eventually, we can remove these Bash scripts from the API code base.
    - Interested users have direct OS access to these scripts allowing them to build their custom PKI solution.
    
    I copied these scripts verbatim from the API and only made minimal changes to make them work:
    - substituted template parameters with script parameters.
    - removed $here variable, as scripts are executed inside the certs dir.
    - removed sign-csr execution from create_device_ca, and instead chained these 2 scripts inside Golang.
    
    The resulting implementation was tested in MEDS factory.
    To test, build the fioctl as this:
    ```
    CGO_ENABLED=0 go build -ldflags "-X=github.com/foundriesio/fioctl/subcommands/version.Commit=v0.36-14-g2cd3815+dirty" -tags bashpki -o bin/fioctl-linux-amd64-bash main.go
    ```
    
    After that, the following command creates PKI using Bash scripts:
    ```
    bin/fioctl-linux-amd64-bash -c fioed-bash.yml keys ca create pki/fioed-bash --local-ca --online-ca
    ```
    
    Maybe, later on we will decide to strip this.
    For now, I prefer to keep this as a backup solution.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    058fae9 View commit details
    Browse the repository at this point in the history
  5. Bugfix: fix HSM support in the Bash PKI implementation

    This is the missing part which rendered the Bash-based PKI support inoperable.
    Now all the pieces seem to be fine.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    8510dbb View commit details
    Browse the repository at this point in the history
  6. Cleanup: change Golang based PKI to use crypto interfaces for keys

    This is a preparation for the HSM support:
    - Use crypto.PublicKey instead of any.
    - User crypto.Signer instead of *ecdsa.PrivateKey.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    12fb5ce View commit details
    Browse the repository at this point in the history
  7. Refactor: extract key and cert read/write PKI routines into separate …

    …module
    
    This makes it easier to introduce the HSM support
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    6f4c845 View commit details
    Browse the repository at this point in the history
  8. Refactor: introduce the PKI storage interface

    This allows to have different key storage implementations.
    This PR immediately adds the filesystem storage for Golang implementation.
    It also adds both filesystem and HSM storage for Bash implementation.
    The pkcs11-tool based implementation is dummy and will be added in the next commit.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    48f4204 View commit details
    Browse the repository at this point in the history
  9. Feature: add HSM support via pkcs11-tool

    This allows to decouple the PKCS11 vendor library loading from the Fioctl process.
    As a result, we can safely continue building static binaries.
    
    This fully replaces all features of our Bash based PKI implementation with the Golang native approach.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    3a79d2a View commit details
    Browse the repository at this point in the history
  10. Feature: add HSM support via PKCS11 dlopen

    This requires dynamic library linking of Libc, hence, not compatible with our static releases.
    But, it was fairly simple to implement, so can be interesting for some users.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    27cf1b4 View commit details
    Browse the repository at this point in the history
  11. Improve: make sure that PKI files are always created read-only

    This fixes two problems:
    - A part of PKI files were created by x509 modules, and part by ca_create.
      This is not only non-uniform, but also makes writing tests harder.
      A small nuissance also goes from different error messages.
    - Because of the above, different files got different permissions.
      This was true for both Bash based on Golang based PKI implementation.
    
    This commit unifies file writing method:
    - It is the x509 package obligation to write files as necessary.
    - All PKI files receive read-only (0400) rights to protect them from unintended access.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    ec849e6 View commit details
    Browse the repository at this point in the history
  12. Improve: require a user to specify the HSM token label

    The default value we provided for this parameter is kind of misleading.
    Furthermore, the user might forget to specify a correct label, resulting into misconfigured HSM module.
    It is safer to require a user to provide the token label explicitly.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    bfe925d View commit details
    Browse the repository at this point in the history
  13. Test: add unit tests for the factory PKI

    These are the first tests for Fioctl, and not something we usually do.
    But, I was rather worried about switching from Bash based to Golang based PKI.
    Especially, given a fact that we need to also migrate the HSM support which was broken for ages.
    
    I also worry that it is way too easy to break a feature which is used so rarely, albeit being so important.
    So, I would prefer these tests to be run as a part of our CI, meaning that this code is being executed regularly.
    
    This first commit adds non-HSM tests.
    As you might notice, it helped me to find a couple of bugs in the Golang based PKI.
    It also helped verifying that the new Golang based PKI is identical to our legacy Bash based PKI.
    Finally, that both implementations work.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    e63c106 View commit details
    Browse the repository at this point in the history
  14. Test: add HSM unit tests for factory PKI

    This test allowed to find few more fixes.
    
    It requires the following packages in order to run:
    - openssl, opensc, softhsm2, libengine-pkcs11-openssl
    
    It does all the necessary initialization and cleanup.
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    556fc29 View commit details
    Browse the repository at this point in the history
  15. Test: add CI workflow for PKI tests

    Make sure to verify all PKI implementations at CI time
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    6badab6 View commit details
    Browse the repository at this point in the history
  16. Improve: cache the pubkey loaded from HSM via pkcs11-tool

    This is a small yet tempting improvement to make a smaller number of calls to the pkcs11-tool.
    It improved the test run time from 0.4 to 0.2 seconds (a negligible diff which simply makes me feel more professional).
    
    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 3, 2023
    Configuration menu
    Copy the full SHA
    88073ab View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. Improve: add a note about pkcs11-tool dependency to README

    Signed-off-by: Volodymyr Khoroz <[email protected]>
    vkhoroz committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    48a08f5 View commit details
    Browse the repository at this point in the history