Python failed to read and write large files

[reclock]

1. Small file reading test is normal

2. A large file (250M), a regular text file, failed to read with the following error. It has been stuck here and has not continued
   

this is my manifest:

fs.mounts = [
{ path = "/lib", uri = "file:{{ gramine.runtimedir() }}" },
{ path = "{{ arch_libdir }}", uri = "file:{{ arch_libdir }}" },
{ path = "/usr/{{ arch_libdir }}", uri = "file:/usr/{{ arch_libdir }}" },
{% for path in python.get_sys_path(entrypoint) %}
{ path = "{{ path }}", uri = "file:{{ path }}" },
{% endfor %}
{ path = "{{ entrypoint }}", uri = "file:{{ entrypoint }}" },

{ type = "tmpfs", path = "/tmp" },
{ path = "/data", uri = "file:/home/gramine-test-app/read-write/wrfile/data/" },
{ path = "/res", uri = "file:/home/gramine-test-app/read-write/wrfile/res/" },
{ path = "/res-enc", uri = "file:/home/gramine-test-app/read-write/wrfile/res-enc/", type = "encrypted", key_name = "_sgx_mrenclave" },
]

sys.enable_extra_runtime_domain_names_conf = true

sgx.debug = true
sgx.edmm_enable = {{ 'true' if env.get('EDMM', '0') == '1' else 'false' }}
sgx.enclave_size = "16G"

sgx.remote_attestation = "dcap"

sgx.trusted_files = [
"file:{{ gramine.libos }}",
"file:{{ entrypoint }}",
"file:{{ gramine.runtimedir() }}/",
"file:{{ arch_libdir }}/",
"file:/usr/{{ arch_libdir }}/",
{% for path in python.get_sys_path(entrypoint) %}
"file:{{ path }}{{ '/' if path.is_dir() else '' }}",
{% endfor %}
"file:scripts/",
]

sgx.allowed_files = [
"file:/home/gramine-test-app/read-write/wrfile/data/",
"file:/home/gramine-test-app/read-write/wrfile/res/",
"file:/home/gramine-test-app/read-write/wrfile/res-enc/",
]

[reclock]

1. Is it feasible for me to run app1 to generate a ciphertext file and then start app2 to read the ciphertext file? App 2 prompts that the ciphertext file cannot be found?
2. Are there any examples of seal and unseal ？

[dimakuv]

1. What is the encryption key with which the ciphertext file is encrypted/decrypted? This key must be shared somehow between app1 and app2. Without knowing more details about your use case, we can't recommend much.
2. Yes, see these:
   • SQLite: here and here
   • Our internal test: here and here

[dimakuv]

I've seen this behavior of Python before. I think what happens in reality has nothing to do with the file read. It is actually a problem of out-of-memory in Python. It's just that Python is known for not handling out-of-memory errors well, so Python can e.g. hang.

Please try to increase sgx.enclave_size to e.g. 32GB. Alternatively, try this setting: sys.brk.max_size = "2G" -- 2GB is just an example, you can try to tweak this value.

Finally, Python is known to require large stacks. By default, Gramine creates stacks of 256KB, so try to increase it e.g. like this: sys.stack.size = "2MB".

[reclock]

I attempted to read a 266M file with the following configuration:

Sys. brk.max_ Size="2G"
Sys. stack. size="1G"
Sgx. enclave_ Size="16G"

But it still cannot be read, why is this?

My machine only has 64GB of memory, so enclave_ Size setting 16G

[dimakuv]

You have weird formatting of the manifest configuration. Are these just typos?

My machine only has 64GB of memory, so enclave_ Size setting 16G

I'm afraid you'll need to find another machine then, to test with sgx.enclave_size = "32G". Alternatively, you can try to set sgx.edmm_enable = true.

As another experiment, try loader.insecure__disable_aslr = true. This effectively provides more memory to the application, but at the cost of security (which is not recommended!).

I'm still quite sure that it is because of insufficient memory.

[reclock]

@dimakuv Sorry, the words have been automatically converted

1. sgx.edmm_enable = true， this configuration has been tested and the driver does not support it. the error:
   error: EDMM feature was requested in manifest, but SGX driver doesn't support it
2. I'm trying to increase memory and verify it again. Thank you very much