9. January 2024

[dimakuv]

Agenda

(please write your proposed agenda items in comments under this discussion)

• Dmitrii & Anees: Continue review of [LibOS,PAL/Linux-SGX] Add EDMM lazy allocation support #1513

  • Performance improvement on certain benchmarks (expecting inputs from @kailun-qin)
  • New PAL API PalGetCommittedPages() -- discuss its implications (split of logic of page management)
  • New LibOS callback get_vma_info(), as part of PalSetMemoryBookkeepingUpcalls() -- discuss its implications (seems benign)
    • This callback/upcall will also help here: [LibOS] When memory pages or access permissions in VMA bookkeeping are not changed, get rid of unnecessary calls to PAL memory ABI or EDMM operations #1708
  • PAL: "enclave page tracker" global object, uses bitvector (embedded in its codebase)
  • New exception case (#PF) in SGX exception handling (_PalExceptionHandler())

• Dmitrii: Reproducible builds, what's the status?

  • Particular requirements: build on Debian, run on MS Azure.
  • Here is our old discussion (which wasn't finished): [Question] Reproducible builds #153

• Dmitrii: Work on https://github.com/orgs/gramineproject/projects/1/views/1 for v1.7

Dmitrii & Anees: Continue review of #1513

• Why MAP_NORESERVE hint?

  • Java and mimalloc use this hint on large allocations (jemalloc seems to also use it)
  • But we need to understand why mimalloc does it, and is it all the time?
    • If all the time, then what's the point of MAP_NORESERVE
  • Michal: maybe remove this hint and just do it always, or never, depending on the new manifest switch
    • Kailun: we didn't want a new manifest option: (1) not to bother users, (2) we thought that this NORESERVE hint is a good compromise between eager allocation by default and lazy allocation with the hint.
    • Michal: also Ok with the current approach (what would be definitely bad: new manifest switch and the hint)

• Dmitrii explained the need for PAL API PalGetCommittedPages()

  • Chia-Che: the name is confusing, should be something else
  • Michal: on Windows it's kinda similar to what we do: reserved and committed memory
  • Don: design choice seems sane; unless you can consolidate memory management in one of LibOS and PAL, you'll have to have this split

• Kailun explained the need for LibOS callback get_vma_info(), as part of PalSetMemoryBookkeepingUpcalls()

  • This is required because during #PF, we want to lazy allocate but we don't have any info about page permissions to allocate with
  • So introduced this callback

• Dmitrii explained the "enclave page tracker" global object

  • Used for freeing only those enclave pages that were allocated
  • SGX PAL has to have this additional state because it's SGX-specific (no sense in making it LibOS-level)

• Michal: no need to split the current PR into smaller PRs, he can review as-is

Dmitrii: Reproducible builds, what's the status?

• Woju: reproducible build must always have some starting point (which binaries you trust). Depending what you trust as your inputs, you can make a "formula" with deterministic inputs and a deterministic toolchain; then the outputs must be also deterministic.
  • (input: Gramine source code) Gramine sources must be reproducible but we didn't check. Meson is configured in a reproducible way. Our subproject libs are already pinned. User must pin the toolchain (the compiler/linker version).
  • (input: Gramine package sources) Gramine package sources must be reproducible.
• Woju: if you have the same dir tree, and you measure it with gramine-sgx-sign twice, you have the same order of files inside the manifest (because Python dicts/lists didn't have an order, but later versions of Python do have an internal order). This we didn't check.
  • Woju: DATE field introduced non-reproducibility, but Woju fixed it recently (incorporated in v1.6). So you get the same SIGSTRUCT twice.
• Woju: long story short: all the pieces are already there, should be reproducible, but nobody really tested. And we want these checks in CI.

[kailun-qin]

Performance improvement on certain benchmarks

• OpenVINO latency tests: bench metrics improved ranging from 2% to 20% (depending on models); execution time improved by ~40 mins (4h 45min w/o this PR; 4h 1min w/ this PR).
• Tensorflow: execution time improved by ~1 hr (7h 53min w/o this PR; 6h 33min w/ this PR).
• OpenJDK example: execution time improved by 32 mins (~33 mins w/o this PR; ~34 secs w/ this PR).

Notes:

1. No perf regressions seen when compared to EDMM enabled w/o this PR.
2. OpenVINO and TensorFlow benchmarks are using mimalloc instead of the default allocator (and the only workloads in our bench set that use mimalloc). The default allocator do not create MAP_NORESERVE mappings at all and thus not benefit from the lazy alloc PR.
3. The absolute execution time captured for OpenVINO and TensorFlow is imprecise (i.e., it's not pure workload execution time but contains some auto framework setup overheads).
4. Our bench set currently does not cover programming language runtimes (e.g., OpenJDK) where mmap(..., MAP_NORESERVE) pattern is usually used.

cc and credit: @vasanth-intel