Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

esp32_sha errors -> 5.6.6-stable-update2-esp32 #7210

Closed
AchimPieters opened this issue Feb 3, 2024 · 56 comments
Closed

esp32_sha errors -> 5.6.6-stable-update2-esp32 #7210

AchimPieters opened this issue Feb 3, 2024 · 56 comments
Assignees

Comments

@AchimPieters
Copy link

AchimPieters commented Feb 3, 2024

Version

5.6.6

Description

Follow-up on #7155

FAILED: esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/port/Espressif/esp32_sha.c.obj 
ccache /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20230928/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc -DESP_PLATFORM -DIDF_VER=\"v5.3-dev-1353-gb3f7e2c8a4\" -DLIBWOLFSSL_VERSION_GIT_BRANCH=\"main\" -DLIBWOLFSSL_VERSION_GIT_HASH=\"eb4487cf7e68dcf34a4a0781d933c8cc1e05c9d7\" -DLIBWOLFSSL_VERSION_GIT_HASH_DATE="\"'Mon Jan 22 17:41:44 2024 +0100'\"" -DLIBWOLFSSL_VERSION_GIT_ORIGIN=\"https://github.com/AchimPieters/esp32-homekit-demo.git\" -DLIBWOLFSSL_VERSION_GIT_SHORT_HASH=\"eb4487c\" -DLIBWOLFSSL_VERSION_WOLFSSL_ROOT=\"/project/components/wolfssl\" -DSOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -DSOC_XTAL_FREQ_MHZ=CONFIG_XTAL_FREQ -DWOLFSSL_USER_SETTINGS_DIR=\"//project//examples//led//components//wolfssl//include//user_settings.h\" -D_GLIBCXX_HAVE_POSIX_SEMAPHORE -D_GLIBCXX_USE_POSIX_SEMAPHORE -D_GNU_SOURCE -D_POSIX_READER_WRITER_LOCKS -I/project/examples/led/build/config -I/project/components/wolfssl/include -I/project/components/wolfssl -I/project/components/wolfssl/wolfssl -I/project/components/wolfssl/wolfssl/wolfcrypt -I/project/components/wolfssl/wolfssl/wolfcrypt/port/Espressif -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/include/freertos -I/opt/esp/idf/components/newlib/platform_include -I/opt/esp/idf/components/freertos/config/include -I/opt/esp/idf/components/freertos/config/include/freertos -I/opt/esp/idf/components/freertos/config/xtensa/include -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/include -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/include -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/include/freertos -I/opt/esp/idf/components/freertos/esp_additions/include -I/opt/esp/idf/components/esp_hw_support/include -I/opt/esp/idf/components/esp_hw_support/include/soc -I/opt/esp/idf/components/esp_hw_support/include/soc/esp32 -I/opt/esp/idf/components/esp_hw_support/port/esp32/. -I/opt/esp/idf/components/heap/include -I/opt/esp/idf/components/log/include -I/opt/esp/idf/components/soc/include -I/opt/esp/idf/components/soc/esp32 -I/opt/esp/idf/components/soc/esp32/include -I/opt/esp/idf/components/hal/platform_port/include -I/opt/esp/idf/components/hal/esp32/include -I/opt/esp/idf/components/hal/include -I/opt/esp/idf/components/esp_rom/include -I/opt/esp/idf/components/esp_rom/include/esp32 -I/opt/esp/idf/components/esp_rom/esp32 -I/opt/esp/idf/components/esp_common/include -I/opt/esp/idf/components/esp_system/include -I/opt/esp/idf/components/esp_system/port/soc -I/opt/esp/idf/components/esp_system/port/include/private -I/opt/esp/idf/components/xtensa/esp32/include -I/opt/esp/idf/components/xtensa/include -I/opt/esp/idf/components/xtensa/deprecated_include -I/opt/esp/idf/components/lwip/include -I/opt/esp/idf/components/lwip/include/apps -I/opt/esp/idf/components/lwip/include/apps/sntp -I/opt/esp/idf/components/lwip/lwip/src/include -I/opt/esp/idf/components/lwip/port/include -I/opt/esp/idf/components/lwip/port/freertos/include -I/opt/esp/idf/components/lwip/port/esp32xx/include -I/opt/esp/idf/components/lwip/port/esp32xx/include/arch -I/opt/esp/idf/components/lwip/port/esp32xx/include/sys -mlongcalls -Wno-frame-address  -DWOLFSSL_USER_SETTINGS -DWOLFSSL_CMAKE_SYSTEM_NAME_LINUX -fdiagnostics-color=always -ffunction-sections -fdata-sections -Wall -Werror=all -Wno-error=unused-function -Wno-error=unused-variable -Wno-error=unused-but-set-variable -Wno-error=deprecated-declarations -Wextra -Wno-unused-parameter -Wno-sign-compare -Wno-enum-conversion -gdwarf-4 -ggdb -Og -fno-shrink-wrap -fmacro-prefix-map=/project/examples/led=. -fmacro-prefix-map=/opt/esp/idf=/IDF -fstrict-volatile-bitfields -fno-jump-tables -fno-tree-switch-conversion -std=gnu17 -Wno-old-style-declaration -MD -MT esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/port/Espressif/esp32_sha.c.obj -MF esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/port/Espressif/esp32_sha.c.obj.d -o esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/port/Espressif/esp32_sha.c.obj -c /project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c: In function 'esp_sha_ctx_copy':
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c:531:12: error: invalid use of undefined type 'struct wc_Sha'
  531 |     if (src->ctx.mode == ESP32_SHA_HW) {
      |            ^~
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c:545:16: error: invalid use of undefined type 'struct wc_Sha'
  545 |             dst->ctx.mode = ESP32_SHA_HW_COPY;
      |                ^~
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c:548:37: error: invalid use of undefined type 'struct wc_Sha'
  548 |             ret = esp_sha_init(&(dst->ctx), WC_HASH_TYPE_SHA);
      |                                     ^~
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c:559:16: error: invalid use of undefined type 'struct wc_Sha'
  559 |         if (dst->ctx.mode == ESP32_SHA_SW) {
      |                ^~
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c:585:12: error: invalid use of undefined type 'struct wc_Sha'
  585 |         dst->ctx.initializer = &(dst->ctx); /* assign new breadcrumb to dst */
      |            ^~
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c:585:37: error: invalid use of undefined type 'struct wc_Sha'
  585 |         dst->ctx.initializer = &(dst->ctx); /* assign new breadcrumb to dst */
      |                                     ^~
/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp32_sha.c:589:16: error: invalid use of undefined type 'struct wc_Sha'
  589 |             dst->ctx.task_owner = 0;
      |                ^~
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the /project/examples/led/build/log/idf_py_stderr_output_41832 and /project/examples/led/build/log/idf_py_stdout_output_41832
root@22fb157df2ff:/project/examples/led# 

For reproduction and latest changes, see: https://github.com/AchimPieters/esp32-homekit-demo

@gojimmypi
Copy link
Contributor

Hello @AchimPieters - I've been able to reproduce the problem you are seeing.

The cause appears to be that with hardware acceleration enabled (by default), the ESP32 libraries do not properly handle this define in your user_settings.h:

#define NO_SHA

Remove that line should fix the compile problem you are seeing.

Additionally, a few other observations and recommendations:

Your example contains an entire copy of wolfSSL. I've (painfully) learned this can be problematic to maintain and troublehoot. The latest versions of the CMakeLists.txt allow for specification of where the wolfSSL source code is located, by setting WOLFSSL_ROOT. See the example template.

There's also a potential problem of distributing wolfSSL. I'm not a licensing expert, so it is probably best to contact [email protected]

It is best to include the wolfSSL settings.h in every file that uses wolfSSL. Include this before referencing any other wolfssl files. Do not explicitly include user_settings.h yourself as the settings.h. pulls it in as appropriate. Later inclusion of the user_settings.h may undo some of the settings.h actions, leading to unpredictable results.

I'm leaving this issue open, as the NO_SHA should be valid regardless of whether hardware encryption is enabled or not.

@AchimPieters
Copy link
Author

@gojimmypi The No SHA did the trick, git some errors that are hardware specific related, but not with Wolfssl.

I'm going to see if I can add wolfssl as a submodule: https://components.espressif.com/components/wolfssl/wolfssl
(License: Custom)

If you have any recommendations, I would like to here them.

@AchimPieters
Copy link
Author

AchimPieters commented Feb 7, 2024

@gojimmypi I solved all problems with the latest version of Wolfssl (v5.6.6-stable), I can compile my code for esp32 but now have this issue?

>>> HomeKit: [Client 1] Pair Setup Step 2/3
>>> homekit_server_on_pair_setup: Free heap: 201056
>>> homekit_server_on_pair_setup: [Client 1] Computing SRP shared secret
>>> homekit_server_on_pair_setup: Free heap: 201056
>>> homekit_server_on_pair_setup: [Client 1] Verifying peer's proof
>>> homekit_server_on_pair_setup: Free heap: 201376
>>> homekit_server_on_pair_setup: [Client 1] Generating own proof
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x34\x35\x0D\x0A\x06\x01\x04\x04\x40\x64\x44\xDC\x7E\xCC\xBC\xFE\xB4\x28\xB8\x9B\x12\x2A\xAA\x1F\x69\x1B\x5E\xF2\x9D\x6F\x7B\x7D\x30\x46\x17\xAE\x4F\xA1\x0B\x2A\x0A\x0A\x8A\x11\x9C\xD1\xD7\xC0\x1F\x08\xEA\xB4\xD1\x0D\xC0\x1E\x63\x22\x75\x84\x44\x63\xF8\xE0\x90\x49\xD5\x29\x59\x60\x50\x39\xBA\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28
>>> homekit_server_accept_client: Free heap: 210620
>>> HomeKit: [Client 1] Got new client connection from 192.168.178.145
>>> homekit_client_process: [Client 1] Got 132 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 211008
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 0 value (1 bytes): \x00
>>> tlv_debug: Type 6 value (1 bytes): \x01
>>> HomeKit: [Client 1] Pair Setup Step 1/3
>>> homekit_server_on_pair_setup: Free heap: 210920
>>> crypto_srp_new: Initializing SRP
>>> homekit_server_on_pair_setup: [Client 1] Initializing crypto
>>> homekit_server_on_pair_setup: Free heap: 201932
>>> homekit_server_on_pair_setup: [Client 1] Using user-specified password: 338-77-883
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Setting SRP verifier
>>> crypto_srp_get_public_key: Calculating public key
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x31\x39\x39\x0D\x0A\x03\xFF\x78\xBD\xF5\x98\xDE\xF6\x3D\x92\x93\x96\xEF\x64\x58\xAF\xB7\x5C\x33\x27\x8D\xFA\x27\x3B\xCA\x6E\x89\x86\x75\x29\xFB\xB1\x55\x90\x7A\x44\x9A\xFE\xD5\x54\x03\x70\x31\xF5\x5D\x64\xBA\xC1\x1B\x33\xE0\xFD\xE6\xDD\x7D\x05\x27\xD0\x36\x1F\xD8\x51\x78\xCD\xE9\x61\x4F\x77\xFF\xEA\x6C\xF4\xA2\x78\x61\xC9\xC9\x16\x86\xE4\xB9\xE4\xA7\x68\xAA\xC6\x06\x1A\x37\x2B\x69\x2B\x83\xE1\xF4\x40\xE4\x01\x08\x4E\x36\xF4\x99\x04\x82\x81\x35\x89\x32\x8D\x0F\x51\x21\xFF\xC7\xCB\x22\x83\x58\x20\x07\xAA\x48\x6B\x54\xAA\x1C\x2D\xDC\xED\xDB\xC7\x25\x90\xB5\x49\xB0\xEB\x7D\x6F\xA7\x4C\xE9\x38\xA7\xC8\xDC\x15\x6C\x4C\xBC\xBA\x23\xDF\xC4\x49\x03\xF9\x22\x33\x5F\x1D\x66\xDB\x41\xB5\x10\x0B\xCE\xE9\x97\x74\x0C\xEB\x65\x02\xA7\x2A\xF6\x09\xCB\xA4\xC8\xF1\x20\x04\xDC\xF1\xBC\x87\x3D\xB5\x72\x87\xFC\x4D\x3D\x1E\x44\x9C\xA4\x83\xA2\x26\x40\x89\xDD\xE1\xF3\x02\x51\x17\xB6\xA3\x7B\x72\x5D\xB2\xA7\xE0\x77\x8E\xF5\xF0\x1A\x82\x08\x46\x89\xCE\x0B\x54\x3E\x37\x29\x54\xB4\x1D\xEE\xB7\x58\x4E\xD3\xBD\x4A\x35\x8E\xE9\x5F\xC2\xBB\xA7\x8D\xE0\x68\xF2\x9A\x03\x81\xD6\xE8\x24\xB4\x0B\x67\xBD\xCC\x00\x46\x05\xB5\x99\x2E\xB5\x80\xFF\x43\x1B\x97\x50\x06\x78\xA0\x0D\xAD\x56\x4F\x83\x59\xD6\x5B\x8E\xF4\x7E\xDF\xB2\x80\x43\x54\xB0\x41\x9A\x40\xEC\x46\xA1\x74\x4C\x71\x19\xF9\x4F\x77\x1B\xAB\x7F\xC6\x1C\xB1\xC1\x20\x05\x1C\x07\x21\xCA\xD2\xE9\x35\x98\xA6\xB5\x5A\x12\xD8\xC2\x07\x4B\x45\xCC\x89\xD4\x60\x49\xB2\x36\x74\x27\x6D\x17\x71\x87\xFF\x4D\x6B\xDC\x5D\xFD\xDC\x3A\x5D\x01\xEF\xAE\xEC\x69\x07\x8B\x3C\xEA\x50\x5A\x49\x7C\x17\x0F\xFB\x43\xD6\x3D\x65\xBF\x47\xF9\x4E\xB1\x99\x18\x02\x10\xE9\x4F\xD8\xCF\xB0\x51\x21\x62\xA5\x12\xDB\xBC\xA0\x25\x34\xB5\x06\x01\x02\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> homekit_client_process: [Client 1] Got 585 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 201604
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 6 value (1 bytes): \x03
>>> tlv_debug: Type 3 value (384 bytes): \xF4\x55\x81\xB5\x54\xB6\x24\x82\x41\x06\xFD\xC0\xE9\x6F\x72\xBD\x64\x65\xF6\xC1\xDA\xE5\x35\xB5\xED\x7D\x64\x91\xB6\x27\x23\x1B\x7F\xC4\x33\x47\xAF\xC1\xA6\x3B\xB7\x0B\xFC\x16\x54\x05\x16\x57\x0F\xFF\xD6\x9B\xD9\xB7\xDE\xD9\x86\xBA\xB2\x75\xDE\xA5\x43\xB2\x51\xB0\xF9\x9E\xCF\x65\x22\x49\xB7\x2D\xAD\x08\x05\xB1\x6E\xF7\xFC\x73\x01\x06\xCA\x00\x56\x98\x37\xB3\xD0\x53\x84\x95\x59\xE5\xE6\x1F\xC0\x80\x0D\x20\x7C\x7E\xFB\xA6\x10\x21\x85\x27\x6E\xEA\x22\x2E\x44\x52\x32\x7F\xAE\x24\x32\xD0\xCF\x11\xE4\x0D\x19\xDC\x2E\x0A\x16\x73\xC3\xD8\x9E\x26\x5C\x8C\x4F\x63\x48\xA6\x3F\x4D\xC0\x9B\x91\x01\xB4\x92\x3E\x81\x12\x34\x44\xFE\x1F\x33\x91\x7F\xFB\x0A\xC3\xC6\xB1\x26\xEF\xCE\x5E\xB7\x96\xF3\xCB\x68\x32\x8D\xAA\x7A\x7A\xC1\x89\x6A\xFF\xEC\x87\x96\xA4\xFB\x61\xE1\xEC\x44\x53\x4F\xDC\x46\x26\x78\xE6\x5E\xD3\x5D\x90\xD6\xD1\xD2\x65\x42\x37\xA9\x7C\xA3\x52\x9A\x82\xDD\x10\x89\x14\xFD\x32\xE8\x22\x09\x35\x98\xBE\x4A\x27\x4F\x0E\xD4\xA9\xDD\x2A\xFA\x24\x08\x42\x8A\x21\xBF\x44\x4E\x7F\xF5\x9F\x58\xBA\x4D\x43\x2B\x49\x97\xD9\xC5\xA4\x9B\xD5\x1C\xB4\xE6\xE8\x9B\x03\xB4\xEE\x3E\x34\xD2\x94\xB0\x36\x93\xB4\x8F\x57\xDA\x87\x5C\x60\xF2\x02\x68\xA9\x9B\xCA\xCA\x03\xEC\x12\x41\xE7\xA0\x9E\xAA\x3A\xEF\xC8\xAC\xA5\x48\x96\xFA\x06\x6B\xD3\x82\x65\x58\xE5\x67\xFE\xBF\xE5\x83\x98\x62\xEC\xCA\xDD\xB4\x8F\xAB\x3B\xFA\x79\xA2\xBF\xFE\x36\x73\x9C\x05\x62\xF5\xB5\xA5\x59\xA5\x69\x99\x30\xF3\xB0\x96\x28\xDF\xA7\xEB\x11\x8C\x9F\x7B\x72\x3A\x17\xC9\xEF\x8F\x7C\x6B\x5F\x87\xE9\xE2\x0A\xD7\xAE\x1C\x5E\x17\x5C\x47\xB3\xD5\x3C\xB8\xEE\x74\xCF\x20\x3A\xF1
>>> tlv_debug: Type 4 value (64 bytes): \x20\x58\xBA\x4A\x7E\x63\xBD\xA2\x7F\xE4\x3F\x2A\x7A\x8A\xCC\x81\x75\x33\x3B\xF4\x04\xAC\x6A\xD9\x49\xAA\x0E\x88\xC6\xFD\x49\xAD\x75\xBF\x46\xFF\xB6\x16\x21\x99\xB4\xE2\x5A\xDD\x4F\x89\xDC\x40\x73\x8B\xFF\xF1\x72\xA2\xE1\x80\xA5\x85\x59\x12\x98\x05\xED\x3A
>>> HomeKit: [Client 1] Pair Setup Step 2/3
>>> homekit_server_on_pair_setup: Free heap: 201036
>>> homekit_server_on_pair_setup: [Client 1] Computing SRP shared secret
>>> homekit_server_on_pair_setup: Free heap: 201036
>>> homekit_server_on_pair_setup: [Client 1] Verifying peer's proof
>>> homekit_server_on_pair_setup: Free heap: 201356
>>> homekit_server_on_pair_setup: [Client 1] Generating own proof
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x34\x35\x0D\x0A\x06\x01\x04\x04\x40\x07\xFC\x7A\xFC\xEF\xDD\x3A\xED\x41\xCE\x15\x53\xB3\x77\xF3\x12\x9F\x3A\x50\x69\x23\x1B\xA9\x19\x51\x84\x63\x89\x41\xEE\x3A\x6E\x4E\x2C\xD9\xD2\x6A\xF5\x84\x64\x4A\xCC\xCF\x9D\xDB\x4B\x94\xA2\x3C\x07\x97\x17\xC8\xF5\xB7\xCC\x05\xF1\x22\xC7\x7B\xCB\x42\x48\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.145

And when compiling for an ESP32C3:

>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Failed to get SRP verifier (code -1)
!!! HomeKit: [Client 1] Failed to initialize SRP
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x36\x0D\x0A\x06\x01\x02\x07\x01\x01\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28

The crypto_srp_init: Failed to get SRP verifier (code -1) error returns? srp errors #6870

latest information can be found here: https://github.com/AchimPieters/esp32-homekit-demo.git

@gojimmypi
Copy link
Contributor

Hello @AchimPieters !

I'm going to see if I can add wolfssl as a submodule ... If you have any recommendations, I would like to here them.

Yes, I really need to put a blog together to better explain that.

First: I'm not so much a fan of using submodules, as least not as a developer. It's great for distribution of a final product but does not lend itself to being able to easily make changes.

Regarding the link you mentioned, that's an Espressif Managed Component. It is a wonderful tool for easily getting started, but that too - does not lend itself to making changes as a developer. In fact, it is even less friendly than a submodule: as soon as a change is detected, it complains. It is great for maintaining code integrity, but not for making changes.

In your example, I deleted everything in this directory except for the CMakeLists.txt file and the include directory:

esp32-homekit-demo\components\wolfssl

I then made one change to the CMakeLists.txt file, pointing to my wolfssl-arduino directory. (I have wolfssl cloned there for Arduino-specific changes).

# Optionally set your source to wolfSSL in your project CMakeLists.txt like this:
set(WOLFSSL_ROOT "c:/workspace/wolfssl-arduino" )

I was able to successfully compile the example LED project that references wolfSSL.

The cool thing here that as a developer, we can more easily see changes and contribute. The library can be shared among as many projects as desired, and versions can easily be changed either with git hashes or even pointing to a completely different directory.

Regarding the more recent SRP errors you are seeing: is that for the LED or some other example? I'm wondering if there's an issue finding the user_settings.h and/or the settings therein. I would need more details.

I might try isolating the project and using wolfSSL as a local component as seen in the examples.

@AchimPieters
Copy link
Author

AchimPieters commented Feb 8, 2024

Hello @gojimmypi,

I was able to successfully compile the example LED project that references wolfSSL.

Also for the esp32C3 or ESPC2?

In your example, I deleted everything in this directory except for the CMakeLists.txt file and the include directory:

Did you do so with my current version, or my old version?

I have tried everything with the latest version of wolffsl 5.6.6-stable-update2-esp32 but noting works any more?

Even the basic ESP32 setup, that worked like a charm, doesn't work any more.
Here is my latest user_settings.h file https://github.com/AchimPieters/esp32-homekit-demo/blob/main/components/wolfssl/include/user_settings.h

Maybe clone my latest repro to see what I mean? https://github.com/AchimPieters/esp32-homekit-demo

@gojimmypi
Copy link
Contributor

Did you do so with my current version, or my old version?

I did this with the same code clone mentioned above, so within the last couple of weeks.

Also for the esp32C3 or ESPC2?

I only tested for the ESP32, but would expect similar successful results with others such the -C3 ad -C2. Did you see different results?

but noting works any more?

I'll need a little more detail, please.

Maybe clone my latest repro to see what I mean?

I'll revisit switch your latest code soon. I just need to finish up a few customer requests.

Any additional information would be helpful, such as specific error messages, please.

@AchimPieters
Copy link
Author

AchimPieters commented Feb 13, 2024

Reproduction steps

Open a terminal window:
Install docker
Install IDF Latest - docker pull espressif/idf:latest
Clone repository - git clone --recursive https://github.com/AchimPieters/esp32-homekit-demo.git
start IDF - docker run -it -v ~/esp32-homekit-demo:/project -w /project espressif/idf:latest
cd examples
cd led
idf.py set-target esp32
idf.py menuconfig

  • Serial flasher config ---> Flash size (2 MB) ---> (X) 4 MB
  • StudioPieters ---> (myssid) WiFi SSID and (mypassword) WiFi Password
  • Component config ---> HomeKit ---> [*] Debug output
    Save configuration
    idf.py build

Open a new (second) terminal window:
cd /Users/USERNAME/esp32-homekit-demo/examples/led
esptool.py erase_flash
python -m esptool --chip esp32 -b 460800 --before default_reset --after hard_reset write_flash --flash_mode dio --flash_size 4MB --flash_freq 40m 0x1000 build/bootloader/bootloader.bin 0x8000 build/partition_table/partition-table.bin 0x10000 build/main.bin

screen /dev/tty.usbserial-01FD1166 115200

I (3838) IP_EVENT: WiFI ready
>>> HomeKit: Starting server
>>> HomeKit: Using existing accessory ID: B9:8B:90:79:57:C7
>>> HomeKit: Configuring mDNS
>>> homekit_setup_mdns: Accessory Setup ID = 1QJ8
>>> homekit_run_server: Starting HTTP server
>>> homekit_server_accept_client: Free heap: 210868
>>> HomeKit: [Client 1] Got new client connection from 192.168.178.28
>>> homekit_client_process: [Client 1] Got 132 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 211256
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 0 value (1 bytes): \x00
>>> tlv_debug: Type 6 value (1 bytes): \x01
>>> HomeKit: [Client 1] Pair Setup Step 1/3
>>> homekit_server_on_pair_setup: Free heap: 211144
>>> crypto_srp_new: Initializing SRP
>>> homekit_server_on_pair_setup: [Client 1] Initializing crypto
>>> homekit_server_on_pair_setup: Free heap: 202168
>>> homekit_server_on_pair_setup: [Client 1] Using user-specified password: 338-77-883
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Setting SRP verifier
>>> crypto_srp_get_public_key: Calculating public key
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x31\x39\x39\x0D\x0A\x03\xFF\xF4\x87\xCB\x6C\x4D\x61\x76\x50\x78\xEA\x36\x9A\x35\x55\x5B\x42\x08\xCE\x0E\xC4\x0A\xE7\x38\x22\xDA\x00\x26\x79\x1B\x76\xA7\xCF\xC4\xB5\x5E\x7D\xF3\xB6\xEF\xD0\x55\xA7\x8D\x21\xFD\x05\x1C\x65\xF8\x19\x08\xEA\xBE\x86\x97\x91\x65\x28\x2C\xE6\x5C\x7D\xB7\x75\xFE\xF8\x66\x2D\x4E\xB1\x53\x7A\x40\xA1\x22\xE6\x3C\xBC\xE2\x3A\x5B\xEC\x49\x84\xC4\x9B\xF5\x93\x62\x95\xAC\x3B\xF6\x3E\x99\x48\x60\x7E\xAD\x3E\xBD\x70\x0E\x10\xC1\x57\xE8\x12\x7C\x38\x36\xCF\x6B\x8E\x95\x2E\x04\x7F\xE0\x30\x82\x11\x2C\x48\x6B\x17\x66\x6F\x5E\x45\x80\x5B\xD3\xA0\xC6\xF9\x9F\x11\xA3\xB8\xC3\x80\xD9\xA1\x24\xDB\xDB\x3D\x2E\xE7\x2D\x14\x06\x12\x02\x2C\x38\x0B\xDD\x0E\x06\xA9\x1D\xBD\xD7\x71\xD8\x0C\x50\xAA\xD2\x94\x2F\x35\x99\x80\x1A\xC0\x93\xAD\x86\x9D\x6A\xC7\x31\xD0\xF9\xCC\xBE\xD2\x66\x14\x15\x4E\xED\x71\x02\xAB\xFC\x8E\xF9\x14\xC1\x9A\xD7\x8A\xB1\xD9\x7A\xC1\xBF\x59\x73\xD1\xA9\x31\xF6\x91\x7C\xE6\xC7\xE8\xF2\xB5\xB0\xE5\x32\x0E\x09\x0A\x9E\x32\xAE\xFB\x90\x06\xC7\xB6\x42\xA6\xA7\x30\xB4\x0B\x16\x53\x17\xC7\x17\xC9\x17\x78\xD3\xF8\x6B\x03\x81\xBB\xEF\x21\x17\xBC\x6D\x59\x1C\x6A\x80\x20\x25\x22\x65\xE3\x9E\x73\xD4\x71\x72\xC6\x83\x39\xCB\x22\x08\x20\xAB\xBD\x71\x8C\x6D\xCA\xC8\x26\x33\x8A\xCD\xDA\xF1\x1D\x73\xDE\xA5\xA9\xAC\x9E\x05\xD3\xBB\xC2\xBB\x3A\x45\x8F\xEE\x38\x12\xE4\x06\xE6\x9E\x47\x8E\xF3\xD1\x85\xC9\xA2\xFA\x84\xB4\x4D\x98\x45\x72\x0B\xE0\x19\x9F\x2D\xE8\x52\xDA\x43\x47\x85\x8F\x5B\xB4\x6E\xC5\x28\x20\x13\x05\x68\x8A\xC4\x5C\x18\x8E\xD9\xD5\xE4\xE8\x29\x64\xB7\x70\x25\x3E\x4A\x7B\xC6\x3B\x9B\xD5\x63\x21\x88\x54\x3C\x92\xD5\x1D\x49\xC0\x12\x02\x10\x2E\x6D\xF4\x69\x1F\x60\x80\x9B\xCA\x46\xDE\xC2\x21\xDA\x5B\x76\x06\x01\x02\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> homekit_client_process: [Client 1] Got 585 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 201856
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 6 value (1 bytes): \x03
>>> tlv_debug: Type 3 value (384 bytes): \x4D\xC4\x55\xBE\x2B\xA6\x56\x28\xAF\x6B\xDA\xAA\x05\x0D\x6E\xDA\xBE\x8C\x5E\x25\x5F\xBD\x0F\x39\x6F\x6B\x68\xB4\xEB\xFD\x5D\xA1\xFD\x50\x4F\xA0\x1A\x6C\xAF\x4E\x94\x38\xB2\xF2\x3C\xAD\x68\x51\x0C\x30\x44\x9A\x7E\x2C\xBA\x31\xF5\x3A\x5E\xEE\x77\xC2\x8A\x43\x75\xC5\x58\xDD\x91\x2A\x16\x48\x27\x5C\x07\xE0\x9A\x0A\xE2\x7E\x79\x11\x33\xBD\x61\xE1\x0D\xB8\x7B\x26\x45\xFC\xED\xAA\x31\x23\x64\xE9\xB7\x28\x73\x0D\x1C\x93\x39\x8C\xCC\x30\x83\x60\x17\x65\x01\xC5\x70\xE5\x3C\xD3\x09\x9C\xE4\xA4\xF4\x09\x99\x58\xC4\x2F\x89\x62\xE7\x02\x7D\x3D\x15\xB7\x5F\x2C\x22\xD6\xCE\x51\x41\x18\x6D\x98\x1B\x78\xA4\x10\x76\xA9\xB2\x7A\x68\xF3\x13\x64\xA2\xB2\x63\x44\xAA\x81\x70\x94\x8A\x83\xD3\xAF\x1D\x8E\xC2\x87\xFA\x4B\xFE\xF3\x78\xE3\x67\xFB\x1E\xC9\x08\xE0\xAA\xDA\xEF\x20\x88\x9B\x9B\x45\x3F\x1B\x5A\xF4\x6E\xED\x7E\x7E\x65\x8F\xC1\x57\xC0\x90\xE9\x5F\xF0\x27\x4A\x21\x1E\x6D\xF6\x10\xEB\xB5\xD6\x83\x76\xF3\xA8\xAA\xCF\xD5\xB1\x62\xF1\xA7\xD5\x82\x02\x9A\x4D\x0E\xD1\xFB\x34\x07\x48\xB8\x32\xF4\xB4\xDF\x75\xB4\x94\xEB\x3E\x5F\xEA\x25\xAF\xC7\xAD\xB1\x40\x79\xCF\xAC\x4F\xA8\x71\xD3\xFC\xB9\x9D\x9C\xB2\x25\x4D\x45\x5A\x31\x7F\x2A\xD1\xF5\x5F\x76\x26\x97\x2B\x2B\xB1\x10\xC6\x84\x0E\x2D\x80\xA2\x42\xEA\xA2\x3F\xEE\x46\x24\x9C\xDC\x13\x18\x9A\xBA\x36\x57\xEC\xB1\x6F\x8E\xA2\x1B\x74\xDD\x94\x1E\xE6\x44\xC0\x4B\x43\x50\x56\xFA\xE8\xF6\xDB\xD7\x41\x58\xAC\x8F\x80\x3C\x83\xDA\x4D\x7A\x80\xCC\xEC\xCE\x33\x07\x41\x3D\xF4\x42\x4C\x34\xC2\x9C\x14\xD1\x15\x40\xC7\x16\x6B\xEC\x4E\x5C\x90\x3E\x0E\x2E\xD1\x80\xB9\xB7\x38\xE4\x1A\xFF\x81\x97\x14\x49\x98
>>> tlv_debug: Type 4 value (64 bytes): \x54\x4D\xBE\xFA\x76\x14\x8D\x15\x79\x2F\x0D\xB6\x19\x55\x5A\x5C\xCE\x11\xE2\x02\xD0\xA5\x34\x9B\x28\x00\xB1\x15\xFC\x6D\xC2\xBB\xEB\xD8\x77\x49\xC8\x47\x16\x13\x30\x3F\xD5\x48\x14\x6F\xA7\xBA\x9C\xC4\x42\x9D\x7B\xDE\x37\xE2\xA2\xBD\x40\xE7\xCA\x41\x81\xBB
>>> HomeKit: [Client 1] Pair Setup Step 2/3
>>> homekit_server_on_pair_setup: Free heap: 201288
>>> homekit_server_on_pair_setup: [Client 1] Computing SRP shared secret
>>> homekit_server_on_pair_setup: Free heap: 201288
>>> homekit_server_on_pair_setup: [Client 1] Verifying peer's proof
>>> homekit_server_on_pair_setup: Free heap: 201516
>>> homekit_server_on_pair_setup: [Client 1] Generating own proof
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x34\x35\x0D\x0A\x06\x01\x04\x04\x40\x0B\x44\x43\x70\x0B\xFD\xBA\xC4\x11\x42\xFF\xD6\x91\xCC\xB1\xBB\x33\xDC\x7D\xB6\xDF\x0A\x8D\x24\x00\xF7\xA8\xE8\x44\xAB\x3D\x38\x0B\xAC\x86\xBE\x71\x25\x13\xF1\xF4\x47\x5D\xF0\xD9\x7F\x62\xCD\xF0\x4E\x8D\x6F\x4B\xF6\x61\x95\x6E\x30\x2A\x66\x9F\x7A\x0C\x39\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28

in HomeKit app -> Unable to add Accessory

@gojimmypi
Copy link
Contributor

Hello @AchimPieters

It looks like you are making excellent progress on your project!

I've reviewed your log, above: it appears wolfSSL is compiling and working properly for you now. What was the problem & solution with your SRP initializer?

Recall I don't have any Apple products, so I am unable to test the HomeKit app with your examples.

The message Unable to add Accessory is rather vague. I'm not sure how I can help you. Is there something with wolfSSL that still needs attention?

@AchimPieters
Copy link
Author

AchimPieters commented Feb 17, 2024

Hello @gojimmypi,

I'm totally lost, can we take it from the top again?

Reproduction steps

Open a terminal window on your mac.

docker pull espressif/idf:latest
  • At this point idf (ESP-IDF v5.3-dev-2032-g4d90eedb6e)
git clone --recursive https://github.com/AchimPieters/esp32-homekit-demo.git
docker run -it -v ~/esp32-homekit-demo:/project -w /project espressif/idf:latest
cd examples/led
idf.py set-target esp32
idf.py menuconfig
  • Select StudioPieters and then (mysid) WIFI SSID and fill in your Wi-Fi Network name, then select (mypassword) WiFI Password and fill in your Wi-Fi Network password.
  • Select component config and then homekit and put a tick at debug output
  • Then press ESC until you are asked Save Configuration? and select (Y)es
idf.py build

And then the errors appear:

it/CMakeFiles/__idf_homekit.dir/src/crypto.c.obj -c /project/components/homekit/src/crypto.c
/project/components/homekit/src/crypto.c:59:19: error: unknown type name 'Srp'
   59 | int wc_SrpSetKeyH(Srp *srp, byte *secret, word32 size) {
      |                   ^~~
/project/components/homekit/src/crypto.c:80:1: error: unknown type name 'Srp'
   80 | Srp *crypto_srp_new() {
      | ^~~
/project/components/homekit/src/crypto.c: In function 'crypto_srp_new':
/project/components/homekit/src/crypto.c:81:9: error: unknown type name 'Srp'
   81 |         Srp *srp = malloc(sizeof(Srp));
      |         ^~~
/project/components/homekit/src/crypto.c:81:34: error: 'Srp' undeclared (first use in this function); did you mean 'srp'?
   81 |         Srp *srp = malloc(sizeof(Srp));
      |                                  ^~~
      |                                  srp
/project/components/homekit/src/crypto.c:81:34: note: each undeclared identifier is reported only once for each function it appears in
/project/components/homekit/src/crypto.c:84:17: error: implicit declaration of function 'wc_SrpInit' [-Werror=implicit-function-declaration]
   84 |         int r = wc_SrpInit(srp, SRP_TYPE_SHA512, SRP_CLIENT_SIDE);
      |                 ^~~~~~~~~~
/project/components/homekit/src/crypto.c:84:33: error: 'SRP_TYPE_SHA512' undeclared (first use in this function); did you mean 'WC_HASH_TYPE_SHA512'?
   84 |         int r = wc_SrpInit(srp, SRP_TYPE_SHA512, SRP_CLIENT_SIDE);
      |                                 ^~~~~~~~~~~~~~~
      |                                 WC_HASH_TYPE_SHA512
/project/components/homekit/src/crypto.c:84:50: error: 'SRP_CLIENT_SIDE' undeclared (first use in this function)
   84 |         int r = wc_SrpInit(srp, SRP_TYPE_SHA512, SRP_CLIENT_SIDE);
      |                                                  ^~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c:89:12: error: request for member 'keyGenFunc_cb' in something not a structure or union
   89 |         srp->keyGenFunc_cb = wc_SrpSetKeyH;
      |            ^~
/project/components/homekit/src/crypto.c:89:30: error: 'wc_SrpSetKeyH' undeclared (first use in this function)
   89 |         srp->keyGenFunc_cb = wc_SrpSetKeyH;
      |                              ^~~~~~~~~~~~~
/project/components/homekit/src/crypto.c: At top level:
/project/components/homekit/src/crypto.c:95:22: error: unknown type name 'Srp'
   95 | void crypto_srp_free(Srp *srp) {
      |                      ^~~
/project/components/homekit/src/crypto.c:101:21: error: unknown type name 'Srp'
  101 | int crypto_srp_init(Srp *srp, const char *username, const char *password) {
      |                     ^~~
/project/components/homekit/src/crypto.c:153:25: error: unknown type name 'Srp'
  153 | int crypto_srp_get_salt(Srp *srp, byte *buffer, size_t *buffer_size) {
      |                         ^~~
/project/components/homekit/src/crypto.c:168:31: error: unknown type name 'Srp'
  168 | int crypto_srp_get_public_key(Srp *srp, byte *buffer, size_t *buffer_size) {
      |                               ^~~
/project/components/homekit/src/crypto.c:187:9: error: unknown type name 'Srp'
  187 |         Srp *srp,
      |         ^~~
/project/components/homekit/src/crypto.c:205:23: error: unknown type name 'Srp'
  205 | int crypto_srp_verify(Srp *srp, const byte *proof, size_t proof_size) {
      |                       ^~~
/project/components/homekit/src/crypto.c:216:26: error: unknown type name 'Srp'
  216 | int crypto_srp_get_proof(Srp *srp, byte *proof, size_t *proof_size) {
      |                          ^~~
/project/components/homekit/src/crypto.c:263:9: error: unknown type name 'Srp'
  263 |         Srp *srp,
      |         ^~~
/project/components/homekit/src/crypto.c: In function 'crypto_chacha20poly1305_decrypt':
/project/components/homekit/src/crypto.c:282:29: error: 'CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE' undeclared (first use in this function)
  282 |         if (message_size <= CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE) {
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c:298:17: error: implicit declaration of function 'wc_ChaCha20Poly1305_Decrypt'; did you mean 'crypto_chacha20poly1305_decrypt'? [-Werror=implicit-function-declaration]
  298 |         int r = wc_ChaCha20Poly1305_Decrypt(
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                 crypto_chacha20poly1305_decrypt
/project/components/homekit/src/crypto.c: In function 'crypto_chacha20poly1305_encrypt':
/project/components/homekit/src/crypto.c:315:37: error: 'CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE' undeclared (first use in this function)
  315 |         size_t len = message_size + CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE;
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c:323:17: error: implicit declaration of function 'wc_ChaCha20Poly1305_Encrypt'; did you mean 'crypto_chacha20poly1305_encrypt'? [-Werror=implicit-function-declaration]
  323 |         int r = wc_ChaCha20Poly1305_Encrypt(
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                 crypto_chacha20poly1305_encrypt
cc1: some warnings being treated as errors
[909/921] Building C object esp-idf/es...s/__idf_espressif__mdns.dir/mdns.c.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the /project/examples/led/build/log/idf_py_stderr_output_471 and /project/examples/led/build/log/idf_py_stdout_output_471
root@6659c5f65e9d:/project/examples/led# 

here is my current unchanged user settings file: https://github.com/AchimPieters/esp32-homekit-demo/blob/main/components/wolfssl/include/user_settings.h

I've tried the "old" fixes, but they resolve the problem "kindisch", maybebe we can have a look specific at the current setup en solve them step by step, to see if this solves the problems?

@gojimmypi
Copy link
Contributor

Hello @AchimPieters

I have been able to replicate the compile problem you are seeing. I appreciate your persistence and patience as we work through these issues.

Try adding these to your wolfssl user_settings.h in esp32-homekit-demo\components\wolfssl\include:

#define WOLFCRYPT_HAVE_SRP
#define HAVE_CHACHA
#define HAVE_POLY1305
#define WOLFSSL_BASE64_ENCODE

Let me know if that works for you.

@AchimPieters
Copy link
Author

@gojimmypi I added the lines and did a compile, everything worked fine until this error?

app partition is too small for binary main.bin is the new version of wolfssl that much bigger then the previous one?

FAILED: esp-idf/esptool_py/CMakeFiles/app_check_size /project/examples/led/build/esp-idf/esptool_py/CMakeFiles/app_check_size 
cd /project/examples/led/build/esp-idf/esptool_py && /opt/esp/python_env/idf5.3_py3.10_env/bin/python /opt/esp/idf/components/partition_table/check_sizes.py --offset 0x8000 partition --type app /project/examples/led/build/partition_table/partition-table.bin /project/examples/led/build/main.bin
Error: app partition is too small for binary main.bin size 0x100720:
  - Part 'factory' 0/0 @ 0x10000 size 0x100000 (overflow 0x720)
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the /project/examples/led/build/log/idf_py_stderr_output_471 and /project/examples/led/build/log/idf_py_stdout_output_471
root@18cf358e7bae:/project/examples/led# 

When changing the partition table to:

# ESP-IDF Partition Table
# Name,   Type, SubType, Offset,  Size,   Flags
nvs,      data, nvs,     0x9000,  0x5000,
phy_init, data, phy,     0xe000,  0x1000,
homekit,  data, homekit, 0xf000,  0x1000,
factory,  app,  factory, 0x10000, 2M,

I can compile without any error, but then when I want to add the accessory to HomeKit it gives this errror:

I (3851) IP_EVENT: WiFI ready
>>> HomeKit: Starting server
>>> HomeKit: Using existing accessory ID: 2B:BC:E2:25:EC:3F
>>> HomeKit: Configuring mDNS
>>> homekit_setup_mdns: Accessory Setup ID = 1QJ8
>>> homekit_run_server: Starting HTTP server
>>> homekit_server_accept_client: Free heap: 210860
>>> HomeKit: [Client 1] Got new client connection from 192.168.178.28
>>> homekit_client_process: [Client 1] Got 132 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 211248
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 0 value (1 bytes): \x00
>>> tlv_debug: Type 6 value (1 bytes): \x01
>>> HomeKit: [Client 1] Pair Setup Step 1/3
>>> homekit_server_on_pair_setup: Free heap: 211136
>>> crypto_srp_new: Initializing SRP
>>> homekit_server_on_pair_setup: [Client 1] Initializing crypto
>>> homekit_server_on_pair_setup: Free heap: 208304
>>> homekit_server_on_pair_setup: [Client 1] Using user-specified password: 338-77-883
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Failed to get SRP verifier (code -1)
!!! HomeKit: [Client 1] Failed to initialize SRP
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x36\x0D\x0A\x06\x01\x02\x07\x01\x01\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28

so the known >>> crypto_srp_init: Failed to get SRP verifier (code -1) as seen here: #6870 (comment)

I could follow the steps above, but want to do this together with you, so we are and stay on the same page.

So what do you recommend doing.

@gojimmypi
Copy link
Contributor

Hi @AchimPieters

app partition is too small for binary main.bin is the new version of wolfssl that much bigger then the previous one?

Well, yes, we turned on new features. We saw that error last year. Curious that you ran out of memory, as I still had 2% available yesterday on my classic ESP32 and did not need to adjust the partition size.

btw - see my defaults, in particular the partitions_singleapp_large.csv that may be helpful.

when I want to add the accessory to HomeKit it gives this errror: crypto_srp_init: Failed to get SRP verifier (code -1)

It looks like the suggested settings from dgarske are missing from your user_settings.h:

If you are building with USE_FAST_MATH (tfc.c) then you need #define FP_MAX_BITS (8192 * 2).
If you are building with WOLFSSL_SP_MATH_ALL (sp_int.c) then set #define SP_INT_BITS 8192

Please add the #define FP_MAX_BITS (8192 * 2) back as before & let me know how it goes.

I suggest keeping USE_FAST_MATH as that's the only wolfSSL math library supporting hardware accelerated encryption on the ESP32 at this time.

Seems you are very close! It would be great to see this all working as desired.

@AchimPieters
Copy link
Author

AchimPieters commented Feb 22, 2024

@gojimmypi

Okay, that did something. Now I still get "Unable to add accessory" and in the terminal:

I (4262) IP_EVENT: WiFI ready
>>> HomeKit: Starting server
>>> HomeKit: Using existing accessory ID: 30:EC:BB:7A:1F:40
>>> HomeKit: Configuring mDNS
>>> homekit_setup_mdns: Accessory Setup ID = 1QJ8
>>> homekit_run_server: Starting HTTP server
>>> homekit_server_accept_client: Free heap: 210832
>>> HomeKit: [Client 1] Got new client connection from 192.168.178.28
>>> homekit_client_process: [Client 1] Got 132 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 211220
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 0 value (1 bytes): \x00
>>> tlv_debug: Type 6 value (1 bytes): \x01
>>> HomeKit: [Client 1] Pair Setup Step 1/3
>>> homekit_server_on_pair_setup: Free heap: 211112
>>> crypto_srp_new: Initializing SRP
>>> homekit_server_on_pair_setup: [Client 1] Initializing crypto
>>> homekit_server_on_pair_setup: Free heap: 202136
>>> homekit_server_on_pair_setup: [Client 1] Using user-specified password: 338-77-883
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Setting SRP verifier
>>> crypto_srp_get_public_key: Calculating public key
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x31\x39\x39\x0D\x0A\x03\xFF\x1E\xCD\xCB\xE9\x02\x7A\x15\x06\xDB\x98\x8C\xC5\xA5\x8A\xCE\x94\xAC\xE7\x9B\xFD\x3E\xCE\x6A\x71\x2E\x09\x61\xC3\xB1\xF9\x26\x70\xEC\x3C\xCA\x81\x0F\x51\x52\x43\x36\x4C\xC2\xB8\x28\x77\x32\xFB\x4B\xBC\x8E\xB5\x26\xE8\x16\xFF\x8E\x67\xA5\x99\x37\x3D\x9D\x20\x9E\xDE\x08\x34\xF0\xB9\x20\xE8\x37\x4E\xE9\x8F\xFE\x0F\x03\xE0\x7A\x44\x4C\x0A\xA5\xFB\x77\xA4\x57\x2B\xC9\xA3\x9B\x17\x96\xFA\x94\xBA\xEA\xCD\x37\x22\x4D\x31\xE0\x80\x34\x8D\x8D\x0F\x88\x84\x38\x76\xCE\x29\xEF\x7C\x1C\x58\xD4\x02\xF5\x6C\x5B\xD5\xF6\x2B\xCF\x0F\x2C\xFC\x49\xDA\x8E\xDB\xAE\xBC\xD3\xF0\xDD\x17\x10\x6E\x3D\xA4\x45\x2C\xF5\x6B\x38\x5D\x48\x3A\x21\xF1\xCB\xEB\xA1\xE4\x05\x06\xFD\xB8\x84\x4D\x60\x9D\xFE\xC4\xEB\x61\x1D\x65\xEB\xA7\xD4\xC1\xD0\x64\x7E\xD4\xF1\x59\xB4\xCE\x9E\x34\x07\xCA\x8E\x0D\x13\x65\xE0\x4E\x73\x94\xFE\xDF\x02\xE7\x86\x88\xFB\xAB\xC4\x9F\xAB\x10\x25\x81\xC9\xDF\x19\x08\x02\xEA\x1B\x0F\xD5\xC7\xC3\x76\x2D\x57\x77\xA5\x29\xBA\x77\x95\x7F\xB0\xC0\xE7\x69\x9F\xFE\x2D\x0F\xDC\x31\x02\x7D\xFA\x95\xC1\xBC\x4C\xD7\xE3\x1C\x94\x9B\x03\x81\x20\xB2\x15\xFD\xBE\xFB\x3B\xD2\xC3\x7A\x9E\xCA\xBF\xE1\x2B\x78\x54\xDA\xB8\xB4\x7D\x51\x53\x3D\xB6\xF2\x98\x19\xBB\xAD\x48\xDB\xE4\x84\xF8\xAA\x38\xDF\x1A\x18\x03\x95\x77\x2B\xD2\x62\xAC\xA0\x6E\x84\xB9\x5E\xE4\xD7\x0B\x38\x4F\x81\x74\xCF\x6F\x11\x91\x9A\xA8\xD8\x08\xAF\x83\x42\xC8\x74\x9A\x77\x82\x98\x18\xFF\x01\x2A\x0F\x17\xCC\xAC\x29\xD4\xF7\xF5\xC2\x7E\x77\x3F\x98\x52\x19\x9F\x89\xFC\x9D\xE1\xA3\x6F\x67\xEE\x6D\x69\x07\x40\xDA\x85\x7A\xCC\x5D\x76\x85\x54\x9C\xCF\x1D\xEC\x63\x3D\x37\xFC\xCF\xC2\xD3\x4E\x10\x02\x10\x7E\x21\xAF\xFD\xDA\x5A\x55\xE5\x8C\x71\x85\x6D\x89\x18\x30\xFF\x06\x01\x02\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> homekit_client_process: [Client 1] Got 585 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 201500
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 6 value (1 bytes): \x03
>>> tlv_debug: Type 3 value (384 bytes): \xC1\x73\xA4\xDF\xCF\x66\xFD\xC3\x74\x21\x5C\x05\xB2\xB7\x47\x30\x9E\xAB\xCA\xA9\x88\x50\x3B\x20\x29\xE9\xCB\x9D\x38\xDD\xA5\x94\x9C\xE3\xE3\x7E\xC5\x28\xD5\x31\xC8\x3E\x8A\x0A\x6D\x8A\xE2\x0B\x1C\x3F\xF7\xEC\x6D\x45\xAD\xDD\x7B\xFF\x7A\xA1\x4A\xA3\x81\xBA\x9B\x7A\xC0\x0C\xF7\x50\x32\xA1\x52\xEE\xF0\x74\x27\x04\x3C\x58\x1B\xD2\xE9\x21\xF4\x64\xDA\x4A\x0A\xCC\x13\x20\xB2\x93\x57\x1B\x5C\xF2\x8B\x94\xDF\x37\x86\x3A\x46\x18\x3F\x47\x05\x98\xA5\x7C\xE3\x41\x33\x21\x1F\xD7\x8E\x4E\xD2\x8C\x55\x20\x4B\x1B\x9F\x29\x8C\x06\xBA\xCC\x76\x82\x44\x94\x53\xE2\xCF\xB1\x88\x1E\x82\x6A\xC9\x96\x5B\xF5\x59\x24\x8E\x6A\x9E\x67\x5A\x2F\xA2\x87\xF4\xE3\x31\x20\xA7\x2D\xF3\x29\x74\xBF\x91\xC0\xFF\x4B\x3C\x51\x13\x27\x25\xE0\xF6\x34\xCC\x5F\x88\xED\x36\x7E\xAA\xB8\x1F\x39\xE9\x21\x92\xED\x03\x49\xCE\xFD\xBA\x38\xD8\x38\x6B\x83\x7E\x5C\xB0\xD3\xAC\x79\x14\x24\xC4\xD3\xCE\x9F\xA0\xC0\x07\x7E\x53\xF3\xD8\xA0\x45\xC7\x8E\xC0\xE2\x71\x84\x04\x25\x39\x17\x40\xD4\xCA\xDF\x85\xD7\x33\x6A\xA3\x4F\xF3\x34\x72\x67\xCA\x2C\x78\x72\xFD\x09\xBB\x79\x4C\x99\x45\x69\xAC\x7E\xAC\xBD\xF2\x5E\x02\x02\x31\x2E\x89\x80\x84\x14\xE2\x02\x47\x41\x93\xE2\xB1\x38\x79\xFB\xE7\x97\x66\xEE\xD2\x68\xDA\x06\x46\x68\x39\x50\xD3\xBA\x7D\x1D\xB5\x87\xF1\x18\xCF\xBA\x76\x13\xAB\x51\xD4\x1D\x5F\x96\x07\x53\x23\xF8\x94\x3E\x0D\xEC\x1D\x20\xD0\x7D\x00\xF4\x8A\x0E\x7D\xB2\x92\x3E\x0E\x0C\xAF\xD9\xE8\x16\x37\xA4\x9B\x1A\x9E\xD7\x54\x18\x20\x38\x97\x30\x40\xAD\x36\xAC\x93\x8F\x41\x00\x1A\x3A\xA1\xAB\xB2\xFD\xCC\xED\xFF\x7A\x5F\xDF\x7C\x97\x98\x89\x46\x6F\x4D\xD3\x6F\x6E\xAE
>>> tlv_debug: Type 4 value (64 bytes): \x29\xD0\x37\xD9\xCE\x1C\x35\x4B\x00\x61\xC1\x75\xD8\xF0\xF6\x4B\x90\x7D\xB0\x32\x35\x77\x7E\xF0\xCF\x56\xBD\x0C\xD4\x2E\xA3\x96\x53\xD9\x87\xBB\x25\x52\xC3\x4F\xEC\x6A\x5C\x7F\x40\x46\x7A\x7E\x6B\xC3\x9E\x0C\x9D\x3B\x84\x88\x39\x30\x1A\xF9\xCB\x20\xB4\x7A
>>> HomeKit: [Client 1] Pair Setup Step 2/3
>>> homekit_server_on_pair_setup: Free heap: 201104
>>> homekit_server_on_pair_setup: [Client 1] Computing SRP shared secret
>>> homekit_server_on_pair_setup: Free heap: 201180
>>> homekit_server_on_pair_setup: [Client 1] Verifying peer's proof
>>> homekit_server_on_pair_setup: Free heap: 201500
>>> homekit_server_on_pair_setup: [Client 1] Generating own proof
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x34\x35\x0D\x0A\x06\x01\x04\x04\x40\x6A\x2C\x00\x97\x57\xF4\x26\x0B\xD5\x58\x1D\x79\xE0\x3B\x5F\x23\xF4\x22\xB5\x85\xEF\xC2\xB3\x72\x52\x16\x8A\x70\x3E\xB8\x41\xC1\x3E\x90\x67\xF9\x2F\x36\x40\x02\x73\x7E\x85\x85\xD6\xEB\x9B\x3A\xB2\xFB\x29\xA6\x18\x96\xA7\x4B\xE2\x37\xAE\x97\x21\x4B\x89\x8B\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28

The SRP works kindich...

Current files and setup: https://github.com/AchimPieters/esp32-homekit-demo.git

@gojimmypi
Copy link
Contributor

Hi @AchimPieters

It seems you are really close! Excellent. What's different from when you last had this working?

It's getting a bit more difficult for me to help, given that I don't have any of the Homekit hardware.

I do have some suggestions though:

  • Try to disable hardware acceleration.

Recently while working on the SSH to UART example, I discovered a problem with SHA256 hardware acceleration as noted in wolfSSL/wolfssh-examples#4. I'm working on a solution to both fix the code, and capture in wolfCrypy test. The root cause was a hash interleave that I did not properly take into account when in SINGLE_THREADED mode when calculating an SSH signature. There could well be a similar problem in the SRP.

The brute-force method to start with is to disable all hardware acceleration in the user_settings.h:

      #define NO_ESP32_CRYPT
      #define NO_WOLFSSL_ESP32_CRYPT_HASH
      #define NO_WOLFSSL_ESP32_CRYPT_AES
      #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI
      #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI_MP_MUL 
      #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI_MULMOD 
      #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI_EXPTMOD
  • On the topic of tests: try using the wolfCrypt test app in your environment with your respective user_settings.h.

  • Try turning on DEBUG_WOLFSSL (see examples for other debug options)

  • Try sprinkling in some printf (or better: ESP_LOGI(TAG, "message")) breadcrumbs to see where it is failing.

  • Try using a JTAG debugger with breakpoints & single-step to find exactly where things go sideways. I use the VisualGDB extension for Visual Studio with the open-source Tigard JTAG debugger. I've found this to be a very helpful and productive tool.

Let me know how that goes. I'd really like to see the your Homekit examples working again.

Best Regards

@AchimPieters
Copy link
Author

@gojimmypi The first test worked, I'm going to do some tests this weekend to see if it also works on a ESPC2-02 and other esp models. I Will let you know, so far so good!

I enabled all these lines in user_settings.h

#define NO_ESP32_CRYPT
   #define NO_WOLFSSL_ESP32_CRYPT_HASH
   #define NO_WOLFSSL_ESP32_CRYPT_AES
   #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI
   #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI_MP_MUL 
   #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI_MULMOD 
   #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI_EXPTMOD

@AchimPieters
Copy link
Author

Tested:

ESP32 WROOM 32D - SUCCES!
ESP32C2 (ESP8684) - Still looking in too...
ESPC3 (ESP-C3-12F Kit):

I (3643) IP_EVENT: WiFI ready
>>> HomeKit: Starting server
>>> HomeKit: Using existing accessory ID: C5:D6:C1:32:8A:B4
>>> HomeKit: Configuring mDNS
>>> homekit_setup_mdns: Accessory Setup ID = 1QJ8
>>> homekit_run_server: Starting HTTP server
>>> homekit_server_accept_client: Free heap: 209740
>>> HomeKit: [Client 1] Got new client connection from 192.168.178.28
>>> homekit_client_process: [Client 1] Got 132 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 211476
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 0 value (1 bytes): \x00
>>> tlv_debug: Type 6 value (1 bytes): \x01
>>> HomeKit: [Client 1] Pair Setup Step 1/3
>>> homekit_server_on_pair_setup: Free heap: 211376
>>> crypto_srp_new: Initializing SRP
>>> homekit_server_on_pair_setup: [Client 1] Initializing crypto
>>> homekit_server_on_pair_setup: Free heap: 208564
>>> homekit_server_on_pair_setup: [Client 1] Using user-specified password: 338-77-883
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Failed to get SRP verifier (code -1)
!!! HomeKit: [Client 1] Failed to initialize SRP
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x36\x0D\x0A\x06\x01\x02\x07\x01\x01\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28

The SRP error is back Failed to get SRP verifier (code -1) ?

@gojimmypi
Copy link
Contributor

Hi @AchimPieters - That's awesome you have it working!

Regarding your ESP-C3-12F Kit: The default hardware acceleration disabling is SoC-specific. Can you please confirm you've disabled it for the ESP32-C3 in your user_settings.h for the CONFIG_IDF_TARGET_ESP32C3?

You may also consider moving the definitions completely outside of the conditional SoC check to disable all hardware acceleration for all devices.

I'll need to look into why the hardware acceleration passes all tests but apparently causes the SRP to fail.

Well done on making such great progress! Cheers

@AchimPieters
Copy link
Author

@gojimmypi, You are awesome, couldn't do it without you! Thank you so much!

Regarding the User_settings.h here is my latest files: https://github.com/AchimPieters/esp32-homekit-demo.git

I'll keep working en testing this weekend, and will share my results!

@AchimPieters
Copy link
Author

@gojimmypi I have checked, double-checked, Krajiceked...

But the SRP verifier (code -1) keeps a mystery.

I also have ordered some ESP32XX Devkits for better debugging capabilities...

@gojimmypi
Copy link
Contributor

Hello @AchimPieters -

Can you please confirm the SRP is fully operational with the hardware acceleration confirmed as disabled for the respective target ESP32 SoC flavor? (I'm not sure I understand the Richard Krajicek tennis link: essentially "quadruple checked"?). Your prior comment seemed to indicate success, but the most recent comment indicates otherwise.

Unless running out of heap or stack, or something wrong with the calling application... I would expect the wolfSSL SRP to be 100% operational in software only mode. (and in theory with hardware acceleration, if everything there is working properly).

I'm assuming this is a hardware acceleration problem, similar to the issue I mentioned with the wolfSSH signature and the interleaved hash calcs not being handled properly. I recently updated the wolfCrypt tests for checking SHA interleaving in #7262.

I believe I've resolved SHA interleave problem with another PR coming soon. It could well be the SRP has a similar or same problem.

I'll be giving this a high priority to resolve. I need to be 100% certain of the results your are seeing, particularly since I don't have the hardware to reproduce myself.

If you are ordering new ESP32 boards: one thing I should have mentioned that in addition to the Tigard external JTAG, the newer Espressif boards with 2 USB ports typically also have JTAG built-in to the board. It is of course not as flexible as having a stand-alone JTAG, but at least that specific board can use the JTAG debugger quite easily.

Thanks for your help. Cheers

@AchimPieters
Copy link
Author

@gojimmypi the SRP is fully operational with the hardware acceleration confirmed as disabled for the respective target ESP32 SoC flavor, But Not for the ESP32C3 and other models.

The Krajicek, was maybe a bad word joke, sorry but that....

ETA for the new ESP boards with double USB ports for better debugging is 7 days, once I receive them, I can do a model by model test. Thank You for your continuous support, you're the best!

@AchimPieters
Copy link
Author

@gojimmypi, I received the modules ESP32, ESP32C2, ESP32C3 and ESP32S3.

As said the ESP32 works, but the other models got SRP errors, I attach a log from every model.
ESP32S2.log
ESP32C3.log
ESP32C2.log
ESP32.log

@gojimmypi
Copy link
Contributor

@AchimPieters Thank you for the additional details and congratulations on your new purchases of other ESP32 devices!

As you said the ESP32 works, but we disabled the hardware acceleration there, right? It appears that your user_settings.h also has hardware disabled for the ESP32-C3. Are you saying that even in software-only mode, you are still seeing SRP errors?

If indeed your are seeing software-only problems, I suggest turning on the stack smashing and heap corruption detection. Perhaps there's a low memory problem. The SRP is fairly robust and unlikely to have problems with hardware acceleration turned off.

I'm actively working on a solution to this & appreciate any additional information.

@AchimPieters
Copy link
Author

@gojimmypi

Are you saying that even in software-only mode, you are still seeing SRP errors?
Yes, as we discussed above, I changed the user_settings.h and there are still SRP errors.

So I'm going to try to follow your advice and turn on the stack smashing and heap corruption detection, and keep you updated.

@gojimmypi
Copy link
Contributor

Hi @AchimPieters

Thanks for the confirmation. I have a fix in the works for the SRP issue. It's my active development branch, but this commit should be relatively good if you'd like to take it for a test drive.

In particular, I have this updated wolfssl_test app that specifically includes a test for the SRP enabled. (and many other features enabled, for the most exhaustive ESP32 test ever). fwiw - I never saw an SRP failure in software-only mode.

It is of course not the same as your actual Homekit app: Perhaps an updated test is in order if you've found something new.

I hope to have a PR together in the relatively near future.

@AchimPieters
Copy link
Author

@gojimmypi Okay I have given it a try, but got this error:

[858/926] Building C object esp-idf/wo...df_wolfssl.dir/wolfcrypt/src/tfm.c.obj
FAILED: esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/tfm.c.obj 
ccache /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20230928/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc -DESP_PLATFORM -DIDF_VER=\"v5.3-dev-2032-g4d90eedb6e\" -DLIBWOLFSSL_VERSION_GIT_BRANCH=\"main\" -DLIBWOLFSSL_VERSION_GIT_HASH=\"1844b49d56e923e612beb14ae11174f0e59f9fc4\" -DLIBWOLFSSL_VERSION_GIT_HASH_DATE="\"'Sun Mar 10 16:22:33 2024 +0100'\"" -DLIBWOLFSSL_VERSION_GIT_ORIGIN=\"https://github.com/AchimPieters/esp32-homekit-demo.git\" -DLIBWOLFSSL_VERSION_GIT_SHORT_HASH=\"1844b49\" -DLIBWOLFSSL_VERSION_WOLFSSL_ROOT=\"/project/components/wolfssl\" -DSOC_MMU_PAGE_SIZE=CONFIG_MMU_PAGE_SIZE -DSOC_XTAL_FREQ_MHZ=CONFIG_XTAL_FREQ -DWOLFSSL_USER_SETTINGS_DIR=\"//project//examples//led//components//wolfssl//include//user_settings.h\" -D_GLIBCXX_HAVE_POSIX_SEMAPHORE -D_GLIBCXX_USE_POSIX_SEMAPHORE -D_GNU_SOURCE -D_POSIX_READER_WRITER_LOCKS -I/project/examples/led/build/config -I/project/components/wolfssl/include -I/project/components/wolfssl -I/project/components/wolfssl/wolfssl -I/project/components/wolfssl/wolfssl/wolfcrypt -I/project/components/wolfssl/wolfssl/wolfcrypt/port/Espressif -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/include/freertos -I/opt/esp/idf/components/newlib/platform_include -I/opt/esp/idf/components/freertos/config/include -I/opt/esp/idf/components/freertos/config/include/freertos -I/opt/esp/idf/components/freertos/config/xtensa/include -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/include -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/include -I/opt/esp/idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/include/freertos -I/opt/esp/idf/components/freertos/esp_additions/include -I/opt/esp/idf/components/esp_hw_support/include -I/opt/esp/idf/components/esp_hw_support/include/soc -I/opt/esp/idf/components/esp_hw_support/include/soc/esp32 -I/opt/esp/idf/components/esp_hw_support/dma/include -I/opt/esp/idf/components/esp_hw_support/port/esp32/. -I/opt/esp/idf/components/esp_hw_support/port/esp32/include -I/opt/esp/idf/components/heap/include -I/opt/esp/idf/components/log/include -I/opt/esp/idf/components/soc/include -I/opt/esp/idf/components/soc/esp32 -I/opt/esp/idf/components/soc/esp32/include -I/opt/esp/idf/components/hal/platform_port/include -I/opt/esp/idf/components/hal/esp32/include -I/opt/esp/idf/components/hal/include -I/opt/esp/idf/components/esp_rom/include -I/opt/esp/idf/components/esp_rom/include/esp32 -I/opt/esp/idf/components/esp_rom/esp32 -I/opt/esp/idf/components/esp_common/include -I/opt/esp/idf/components/esp_system/include -I/opt/esp/idf/components/esp_system/port/soc -I/opt/esp/idf/components/esp_system/port/include/private -I/opt/esp/idf/components/xtensa/esp32/include -I/opt/esp/idf/components/xtensa/include -I/opt/esp/idf/components/xtensa/deprecated_include -I/opt/esp/idf/components/lwip/include -I/opt/esp/idf/components/lwip/include/apps -I/opt/esp/idf/components/lwip/include/apps/sntp -I/opt/esp/idf/components/lwip/lwip/src/include -I/opt/esp/idf/components/lwip/port/include -I/opt/esp/idf/components/lwip/port/freertos/include -I/opt/esp/idf/components/lwip/port/esp32xx/include -I/opt/esp/idf/components/lwip/port/esp32xx/include/arch -I/opt/esp/idf/components/lwip/port/esp32xx/include/sys -mlongcalls -Wno-frame-address  -DWOLFSSL_USER_SETTINGS -DWOLFSSL_CMAKE_SYSTEM_NAME_LINUX -fdiagnostics-color=always -ffunction-sections -fdata-sections -Wall -Werror=all -Wno-error=unused-function -Wno-error=unused-variable -Wno-error=unused-but-set-variable -Wno-error=deprecated-declarations -Wextra -Wno-unused-parameter -Wno-sign-compare -Wno-enum-conversion -gdwarf-4 -ggdb -Og -fno-shrink-wrap -fmacro-prefix-map=/project/examples/led=. -fmacro-prefix-map=/opt/esp/idf=/IDF -fstrict-volatile-bitfields -fno-jump-tables -fno-tree-switch-conversion -std=gnu17 -Wno-old-style-declaration -MD -MT esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/tfm.c.obj -MF esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/tfm.c.obj.d -o esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/tfm.c.obj -c /project/components/wolfssl/wolfcrypt/src/tfm.c
/project/components/wolfssl/wolfcrypt/src/tfm.c: In function '_fp_exptmod_nct':
/project/components/wolfssl/wolfcrypt/src/tfm.c:2467:6: error: implicit declaration of function 'WOLFSSL_MSG_EX' [-Werror=implicit-function-declaration]
 2467 |      WOLFSSL_MSG_EX("_fp_exptmod_nct XMALLOC failed: %d", (word32)sizeof(fp_int)*((1 << winsize) + 1));
      |      ^~~~~~~~~~~~~~
/project/components/wolfssl/wolfcrypt/src/tfm.c: In function 'fp_exptmod_nct':
/project/components/wolfssl/wolfcrypt/src/tfm.c:3313:7: error: implicit declaration of function 'WOLFSSL_MSG'; did you mean 'WOLFSSL_API'? [-Werror=implicit-function-declaration]
 3313 |       WOLFSSL_MSG("handle modulus of zero");
      |       ^~~~~~~~~~~
      |       WOLFSSL_API
cc1: some warnings being treated as errors
[863/926] Building C object esp-idf/wo...df_wolfssl.dir/wolfcrypt/src/srp.c.obj
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the /project/examples/led/build/log/idf_py_stderr_output_713 and /project/examples/led/build/log/idf_py_stdout_output_713
root@5afeccbad994:/project/examples/led# 

I don't know whether I did something wrong or something wrong with your branch?

@gojimmypi
Copy link
Contributor

Hi @AchimPieters

My branch is under active development so at some point, there might have been inoperable code at some point.

When WOLFSSL_MSG and/or WOLFSSL_MSG_EX is missing, simply include the wolfSSL logging header:

#include <wolfssl/wolfcrypt/logging.h>

The wolfcrypt TFM library does not include this by default. I must have been testing something. Feel free to add the include or simply comment out those lines. They are only interesting with #define DEBUG_WOLFSSL in the user_settings.h and turned on with wolfSSL_Debugging_ON();

My latest branch update is working well with SRP and includes an improved logging messages when heap allocation fails. It would be great for you to take it for a test drive.

@gojimmypi
Copy link
Contributor

Hi @AchimPieters - thank you so much for providing the user_settings.h and for detailing the ESP32 works and the ESP32-C2 does not work with SRP.

The first thing that comes to mind with one device working but not another is your memory configuration: Note that the C2 has less RAM: 272 KB of SRAM (16 KB dedicated to cache) *1 vs the ESP32 that has 520 KB SRAM *2.

Could you please provide all of the settings in your sdkconfig.h that include the text stack, heap, and IRAM. You might also consider turning on stack smashing and/or heap corruption detection.

Also, I didn't see any setting for session caching. See the user_settings.h for the TLS Client Example. Smaller cache can save memory.

#define SMALL_SESSION_CACHE

That said, the most recent wolfSSL v5.6.0 does not include all of my changes. One thing in particular with is SHA interleaving improvements that I am still testing. I'm not referring to the HW interleave capability, but rather the current HW implementation with multiple, concurrent SHA calculations that typically occur in something such as SRP. I recently added #7262 that helps detect this. See the Espressif-specific wolfssl_test example.

@AchimPieters
Copy link
Author

AchimPieters commented Apr 1, 2024

@gojimmypi Here is the complete sdkconfig.h sdkconfig.txt

Turning on stack smashing and/or heap corruption detection, didn't provide any useful information.

I added Smaller cache: user_settings.txt but gave me the same error:

 >>> homekit_server_accept_client: Free heap: 89624
>>> HomeKit: [Client 1] Got new client connection from 192.168.178.28
>>> homekit_client_process: [Client 1] Got 132 incoming data
>>> homekit_server_on_pair_setup: Pair Setup
>>> homekit_server_on_pair_setup: Free heap: 91344
>>> tlv_debug: Got following TLV values:
>>> tlv_debug: Type 0 value (1 bytes): \x00
>>> tlv_debug: Type 6 value (1 bytes): \x01
>>> HomeKit: [Client 1] Pair Setup Step 1/3
>>> homekit_server_on_pair_setup: Free heap: 91252
>>> crypto_srp_new: Initializing SRP
>>> homekit_server_on_pair_setup: [Client 1] Initializing crypto
>>> homekit_server_on_pair_setup: Free heap: 88492
>>> homekit_server_on_pair_setup: [Client 1] Using user-specified password: 338-77-883
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> tlv_debug: Type 0 value (1 bytes): \x00
>>> tlv_debug: Type 6 value (1 bytes): \x01
>>> HomeKit: [Client 1] Pair Setup Step 1/3
>>> homekit_server_on_pair_setup: Free heap: 91252
>>> crypto_srp_new: Initializing SRP
>>> homekit_server_on_pair_setup: [Client 1] Initializing crypto
>>> homekit_server_on_pair_setup: Free heap: 88492
>>> homekit_server_on_pair_setup: [Client 1] Using user-specified password: 338-77-883
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Failed to get SRP verifier (code -1)
!!! HomeKit: [Client 1] Failed to initialize SRP
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x36\x0D\x0A\x06\x01\x02\x07\x01\x01\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28

@gojimmypi
Copy link
Contributor

gojimmypi commented Apr 6, 2024

Hello @AchimPieters

TL;DR: The version of wolfSSL you are using does not have my SRP fix. Try this one.

Here are my notes:

Heads up the Icon file is still problematic on Visual Studio / Windows:

C:\workspace>git clone --recursive https://github.com/AchimPieters/esp32-homekit-demo.git esp32-homekit-demo-gojimmypi
Cloning into 'esp32-homekit-demo-gojimmypi'...
remote: Enumerating objects: 4725, done.
remote: Counting objects: 100% (359/359), done.
remote: Compressing objects: 100% (257/257), done.
remote: Total 4725 (delta 136), reused 166 (delta 98), pack-reused 4366Receiving objects: 100% (4725/4725), 39.09 MiB | Receiving objects: 100% (4725/4725), 40.87 MiB | 6.44 MiB/s, done.

Resolving deltas: 100% (1815/1815), done.
error: invalid path 'Icon?'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

I tried to submit a request on the Visual Studio Developer Community without much luck.

When using Linux and adding an upstream like this:

git remote add upstream https://github.com/AchimPieters/esp32-homekit-demo.git

I saw this remote error: upload-pack: not our ref error:


gojimmypi:/mnt/c/workspace/esp32-homekit-demo-gojimmypi
$ git fetch upstream main
remote: Enumerating objects: 997, done.
remote: Counting objects: 100% (390/390), done.
remote: Compressing objects: 100% (199/199), done.
remote: Total 997 (delta 212), reused 255 (delta 187), pack-reused 607
Receiving objects: 100% (997/997), 15.65 MiB | 2.25 MiB/s, done.
Resolving deltas: 100% (463/463), completed with 90 local objects.
From https://github.com/AchimPieters/esp32-homekit-demo
 * branch            main       -> FETCH_HEAD
 * [new branch]      main       -> upstream/main
Fetching submodule components/homekit
fatal: remote error: upload-pack: not our ref 80ae7716b49e7005c441a3286a6a56af8fb42aff
Errors during submodule fetch:
        components/homekit

So I ended up having to delete the entire repo and forking a fresh one.

Ha! Today I also learned one cannot have two forks of the same repo, even if the first fork is renamed.

In an any case, my new fork of esp32-homekit-demo seems to be much happier.

Upon inspection, the first thing I noticed is that the esp32-homekit-demo wolfssl component appears to be a copy of the wolfSSL v5.7.0 Espressif Managed Component.

Please note that not only have my SRP changes mentioned above not made it into the v5.7.0 release, I've not even yet created a PR to merge them into wolfssl.

The good news, is that my 9x SRP testing. (all passing) commit at commit 6f92cc9 appears to be passing the SRP test for all flavors of ESP32 on my testing jig including an ESP8266 that I later added to the center.

See the wolfssl_test app at that point in history on my fork.

How to proceed? For me, I'd clone wolfssl at the root level of the project, with the same parent directory as images, Documentation, etc. Be sure to remove the existing one.

Then in the components directory, only a few files are needed as seen in the example:

  • components/CMakeLists.txt
  • components/include/user_settings.h

The CMakeLists.txt has a feature that searches for wolfssl or you can set an environment variable called WOLFSSL_ROOT pointing to the root of wolfssl.

Once you plug in my commit 6f92cc9 (6f92cc98968d03baf14aaa54a8e6919d60365347) version, I'm very curious as to the results you see.

The big thing again is memory. You'll see in my user_settings.h I added a sanity check at the end:

#if defined(CONFIG_ESP_MAIN_TASK_STACK_SIZE)
    #if defined(WOLFCRYPT_HAVE_SRP)
        #if defined(FP_MAX_BITS)
            #if FP_MAX_BITS <  (8192 * 2)
                #define ESP_SRP_MINIMUM_STACK_8K (24 * 1024)
            #else
                #define ESP_SRP_MINIMUM_STACK_8K (28 * 1024)
            #endif
        #else
            #error "Please define FP_MAX_BITS when using WOLFCRYPT_HAVE_SRP."
        #endif

        #if (CONFIG_ESP_MAIN_TASK_STACK_SIZE < ESP_SRP_MINIMUM_STACK)
            #warning "WOLFCRYPT_HAVE_SRP enabled with small stack size"
        #endif
    #endif
#else
    #warning "CONFIG_ESP_MAIN_TASK_STACK_SIZE not defined!"
#endif

The 28K is a very bare-minimum amount for the wolfssl_test exercise. Any other application would likely need
additional overhead.

Here's the output from my test script showing the SRP passing on all devices:

image

Note I've also added some additional sdkconfig.default.[device] files that adjusts the memory as appropriate.

Let me know if you are able to test with my version of wolfSSL.

I am certain we can get your Apple Homekit project working on every flavor of ESP32.

Thank you again for your patience and persistence. Cheers.

@AchimPieters
Copy link
Author

@gojimmypi thank you for the extensive work you have done, at the moment I am ill (Notting serious), but gave it a quick go: you can find the branch here: https://github.com/AchimPieters/esp32-homekit-demo/tree/RSP-WOLFSSL

But now I got these errors:

In file included from /project/components/wolfssl/wolfssl/wolfcrypt/setIn file included from /project/components/wolfssl/wolfssl/wolfcrypt/settings.h:301,
                 from /project/components/homekit/src/crypto.c:3:
/project/components/wolfssl/include/user_settings.h:541:22: warning: #warning "RSA may be difficult with less than 10KB Stack "/ [-Wcpp]
  541 |                     #warning "RSA may be difficult with less than 10KB Stack "/
      |                      ^~~~~~~
In file included from /project/components/homekit/src/crypto.c:4:
/project/components/wolfssl/include/user_settings.h:541:22: warning: #warning "RSA may be difficult with less than 10KB Stack "/ [-Wcpp]
  541 |                     #warning "RSA may be difficult with less than 10KB Stack "/
      |                      ^~~~~~~
/project/components/homekit/src/crypto.c: In function 'crypto_chacha20poly1305_decrypt':
/project/components/homekit/src/crypto.c:282:29: error: 'CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE' undeclared (first use in this function)
  282 |         if (message_size <= CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE) {
      |                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c:282:29: note: each undeclared identifier is reported only once for each function it appears in
/project/components/homekit/src/crypto.c:298:17: error: implicit declaration of function 'wc_ChaCha20Poly1305_Decrypt'; did you mean 'crypto_chacha20poly1305_decrypt'? [-Werror=implicit-function-declaration]
  298 |         int r = wc_ChaCha20Poly1305_Decrypt(
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                 crypto_chacha20poly1305_decrypt
/project/components/homekit/src/crypto.c: In function 'crypto_chacha20poly1305_encrypt':
/project/components/homekit/src/crypto.c:315:37: error: 'CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE' undeclared (first use in this function)
  315 |         size_t len = message_size + CHACHA20_POLY1305_AEAD_AUTHTAG_SIZE;
      |                                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c:323:17: error: implicit declaration of function 'wc_ChaCha20Poly1305_Encrypt'; did you mean 'crypto_chacha20poly1305_encrypt'? [-Werror=implicit-function-declaration]
  323 |         int r = wc_ChaCha20Poly1305_Encrypt(
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                 crypto_chacha20poly1305_encrypt
/project/components/homekit/src/crypto.c: In function 'crypto_curve25519_init':
/project/components/homekit/src/crypto.c:466:17: error: implicit declaration of function 'wc_curve25519_init'; did you mean 'wc_ed25519_init'? [-Werror=implicit-function-declaration]
  466 |         int r = wc_curve25519_init(key);
      |                 ^~~~~~~~~~~~~~~~~~
      |                 wc_ed25519_init
/project/components/homekit/src/crypto.c: In function 'crypto_curve25519_done':
/project/components/homekit/src/crypto.c:478:9: error: implicit declaration of function 'wc_curve25519_free'; did you mean 'wc_ed25519_free'? [-Werror=implicit-function-declaration]
  478 |         wc_curve25519_free(key);
      |         ^~~~~~~~~~~~~~~~~~
      |         wc_ed25519_free
/project/components/homekit/src/crypto.c: In function 'crypto_curve25519_generat':
/project/components/homekit/src/crypto.c:496:13: error: implicit declaration of function 'wc_curve25519_make_key'; did you mean 'wc_ed25519_make_key'? [-Werror=implicit-function-declaration]
  496 |         r = wc_curve25519_make_key(&rng, 32, key);
      |             ^~~~~~~~~~~~~~~~~~~~~~
      |             wc_ed25519_make_key
/project/components/homekit/src/crypto.c: In function 'crypto_curve25519_import_public':
/project/components/homekit/src/crypto.c:507:16: error: implicit declaration of function 'wc_curve25519_import_public_ex'; did you mean 'wc_ed25519_import_public_ex'? [-Werror=implicit-function-declaration]
  507 |         return wc_curve25519_import_public_ex(data, size, key, EC25519_LITTLE_ENDIAN);
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                wc_ed25519_import_public_ex
/project/components/homekit/src/crypto.c:507:64: error: 'EC25519_LITTLE_ENDIAN' undeclared (first use in this function)
  507 |         return wc_curve25519_import_public_ex(data, size, key, EC25519_LITTLE_ENDIAN);
      |                                                                ^~~~~~~~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c: In function 'crypto_curve25519_export_public':
/project/components/homekit/src/crypto.c:514:25: error: implicit declaration of function 'wc_curve25519_export_public_ex'; did you mean 'wc_ed25519_import_public_ex'? [-Werror=implicit-function-declaration]
  514 |                 int r = wc_curve25519_export_public_ex(
      |                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                         wc_ed25519_import_public_ex
/project/components/homekit/src/crypto.c:517:25: error: 'EC25519_LITTLE_ENDIAN' undeclared (first use in this function)
  517 |                         EC25519_LITTLE_ENDIAN
      |                         ^~~~~~~~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c: In function 'crypto_curve25519_shared_secret':
/project/components/homekit/src/crypto.c:535:21: error: 'CURVE25519_KEYSIZE' undeclared (first use in this function); did you mean 'ED25519_KEY_SIZE'?
  535 |         if (*size < CURVE25519_KEYSIZE) {
      |                     ^~~~~~~~~~~~~~~~~~
      |                     ED25519_KEY_SIZE
/project/components/homekit/src/crypto.c:541:17: error: implicit declaration of function 'wc_curve25519_shared_secret_ex'; did you mean 'crypto_curve25519_shared_secret'? [-Werror=implicit-function-declaration]
  541 |         int r = wc_curve25519_shared_secret_ex(
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                 crypto_curve25519_shared_secret
/project/components/homekit/src/crypto.c:544:31: error: 'EC25519_LITTLE_ENDIAN' undeclared (first use in this function)
  544 |                 buffer, &len, EC25519_LITTLE_ENDIAN
      |                               ^~~~~~~~~~~~~~~~~~~~~
/project/components/homekit/src/crypto.c: In function 'crypto_curve25519_import_public':
/project/components/homekit/src/crypto.c:508:1: error: control reaches end of non-void function [-Werror=return-type]
  508 | }
      | ^
cc1: some warnings being treated as errors
ninja: build stopped: subcommand failed.
ninja failed with exit code 1, output of the command is in the /project/examples/led/build/log/idf_py_stderr_output_9900 and /project/examples/led/build/log/idf_py_stdout_output_9900
root@48772da10d14:/project/examples/led# 

Maybe It will give you an idea right away, but for me, I have to put it on hold, until i'm better.

@AchimPieters
Copy link
Author

AchimPieters commented Apr 7, 2024

@gojimmypi did some tweaking to user_settings.h and changed the sdkconfig files as suggested.
Here are my latest files: https://github.com/AchimPieters/esp32-homekit-demo/tree/RSP-WOLFSSL

But now I got a real strange one when compiling for esp32:

/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp_sdk_wifi_lib.c:37:10: fatal error: esp_wifi.h: No such file or directory
   37 | #include <esp_wifi.h>
      |          ^~~~~~~~~~~~
compilation terminated.
[869/921] Building C object esp-idf/wolfssl/CMakeFiles/_....dir/wolfcrypt/src/port/Espressif/esp_sdk_time_lib.c.ob
FAILED: esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/port/Espressif/esp_sdk_time_lib.c.obj 

And

/project/components/wolfssl/wolfcrypt/src/port/Espressif/esp_sdk_time_lib.c:62:18: fatal error: esp_netif_sntp.h: No such file or directory
   62 |         #include <esp_netif_sntp.h>
      |                  ^~~~~~~~~~~~~~~~~~
compilation terminated.
[872/921] Building C object esp-idf/wolfssl/CMakeFiles/__idf_wolfssl.dir/wolfcrypt/src/tfm.c.obj
ninja: build stopped: subcommand failed.
Compilation failed because esp_sdk_wifi_lib.c (in "wolfssl" component) includes esp_wifi.h, provided by esp_wifi component(s).
However, esp_wifi component(s) is not in the requirements list of "wolfssl".
To fix this, add esp_wifi to PRIV_REQUIRES list of idf_component_register call in /project/components/wolfssl/CMakeLists.txt.
ninja failed with exit code 1, output of the command is in the /project/examples/led/build/log/idf_py_stderr_output_7752 and /project/examples/led/build/log/idf_py_stdout_output_7752
root@e3b704ffe19d:/project/examples/led# 

But when I'm looking into the source directory's: the /components/wolfssl/wolfcrypt/src/port/Espressif/esp_sdk_wifi_lib AND the components/wolfssl/wolfcrypt/src/port/Espressif/esp_sdk_time_lib.c is also there?
Screenshot 2024-04-07 at 10 50 05

I don't want to fiddle too much, because otherwise we don't remember what we did, don't you agree?

Thanks for spending time on making a robust design and writing code and tests with best practices. It took you some extra effort, but this will benefit the product in the long run.

@gojimmypi
Copy link
Contributor

Hi @AchimPieters sorry you are not well. I hope you are feeling better soon.

But now I got these errors

Ah yes. One of these days I will improve the user_settings.h sanity checks. The errors you see are from an undesired user settings file.

fatal error: esp_wifi.h: No such file or directory #include <esp_wifi.h>

I was working on this a bit yesterday & saw the same message. This is related to a new feature moving some code like wifi and time to wolfssl helpers.

It seems that when the wolfSSL component is in a different directory than the project being built, there's a problem finding the Espressif dependency.

I've pushed some new code that disables the new libraries by default.

"A code segment is work a thousand words". Let's try this:

# Setup your ESP-IDF environment as needed.
# Any ESP-IDF v5.2 should work. This one shown for my WSL using VisualGDB:
WRK_IDF_PATH=/mnt/c/SysGCC/esp32/esp-idf/v5.2
. ${WRK_IDF_PATH}/export.sh

# Fetch a copy of esp32-homekit-demo
git clone --recursive https://github.com/gojimmypi/esp32-homekit-demo.git esp32-homekit-demo-gojimmypi
cd  ./esp32-homekit-demo-gojimmypi

# Optionally set upstream
git remote add upstream https://github.com/AchimPieters/esp32-homekit-demo.git

# Fetch a copy of the gojimmypi wolfssl fork into current directory (from ./esp32-homekit-demo-gojimmypi)
git clone https://github.com/gojimmypi/wolfssl.git

# Checkout the desired commit:
cd wolfssl
git checkout a84feaba2454b4262f29dd9b1ee80465d00114c7
cd ..

# Still in esp32-homekit-demo-gojimmypi directory.
# Remove the existing component source for the published 5.7.0 version:
rm -rf ./components/wolfssl/examples
rm -rf ./components/wolfssl/src
rm -rf ./components/wolfssl/wolfssl
rm -rf ./components/wolfssl/wolfcrypt
rm     ./components/wolfssl/.component_hash
rm     ./components/wolfssl/idf_component.yml

# Get a recent version of wolfssl component CMake file:
cp ./wolfssl/IDE/Espressif/ESP-IDF/examples/template/components/wolfssl/CMakeLists.txt ./components/wolfssl/CMakeLists.txt

# Optionally set location of wolfSSL. The location of the clone in the parent should be found even without this:
# WOLFSSL_ROOT=/mnt/c/workspace/esp32-homekit-demo-gojimmypi/wolfssl

cd ./examples/led
idf.py set-target esp32
idf.py menuconfig
idf.py build

The important things here are:

  • My branch of wolfssl (PR coming soon).
  • The recent ./esp32-homekit-demo-gojimmypi/components/wolfssl/CMakeLists.txt file.
  • Your original ./esp32-homekit-demo-gojimmypi/components/wolfssl/include/user_ettings.h file.

It is also important to ensure the calling freeRTOS stack (or your main app) has enough stack configured for SRP.

I was thinking it might be useful (and a common testing ground, since I don't have any Apple products), if you'd like to try calling the wolfSSL test right before your connection test?

ret = wolf_test_task();

When you get a chance, please give this a try and let me know how it goes.

btw, while compiling I saw this "flash nearly full" warning:

Successfully created esp32 image.
Generated /mnt/c/test/temp/esp32-homekit-demo-gojimmypi/examples/led/build/main.bin
[7/7] cd /mnt/c/test/temp/esp32-homekit-demo-gojimmypi/exa...p/esp32-homekit-demo-gojimmypi/examples/led/build/main.bin
main.bin binary size 0xfb000 bytes. Smallest app partition is 0x100000 bytes. 0x5000 bytes (2%) free.
Warning: The smallest app partition is nearly full (2% free space left)!

Thanks again for your help on this. Your project structure has definitely pointed out some weaknesses in my settings and cmake files.

Cheers

@AchimPieters
Copy link
Author

AchimPieters commented Apr 8, 2024

@gojimmypi Here are what I did and the results:

git clone --recursive https://github.com/gojimmypi/esp32-homekit-demo.git esp32-homekit-demo-gojimmypi
cd ./esp32-homekit-demo-gojimmypi

git remote add upstream https://github.com/AchimPieters/esp32-homekit-demo.git

git clone https://github.com/gojimmypi/wolfssl.git

cd wolfssl
git checkout a84feaba2454b4262f29dd9b1ee80465d00114c7
cd ..

rm -rf ./components/wolfssl/examples
rm -rf ./components/wolfssl/src
rm -rf ./components/wolfssl/wolfssl
rm -rf ./components/wolfssl/wolfcrypt
rm     ./components/wolfssl/.component_hash
rm     ./components/wolfssl/idf_component.yml

cp ./wolfssl/IDE/Espressif/ESP-IDF/examples/template/components/wolfssl/CMakeLists.txt ./components/wolfssl/CMakeLists.txt

ESP32

cd ./examples/led
idf.py set-target esp32
idf.py menuconfig
idf.py build

  • Compling
  • Connecting to iPhone
  • Overal function

ESP32C2

cd ./examples/led
idf.py set-target esp32C2
idf.py menuconfig
idf.py build

  • Compling
  • Connecting to iPhone
  • Overal function
>>> crypto_srp_init: Generating salt
>>> crypto_srp_init: Setting SRP username
>>> crypto_srp_init: Setting SRP params
>>> crypto_srp_init: Setting SRP password
>>> crypto_srp_init: Getting SRP verifier
>>> crypto_srp_init: Failed to get SRP verifier (code -1)
!!! HomeKit: [Client 1] Failed to initialize SRP
>>> client_sendv: [Client 1] Sending payload: HTTP/1.1 200 OK\x0D\x0AContent-Type: application/pairing+tlv8\x0D\x0ATransfer-Encoding: chunked\x0D\x0AConnection: keep-alive\x0D\x0A\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x36\x0D\x0A\x06\x01\x02\x07\x01\x01\x0D\x0A
>>> client_sendv: [Client 1] Sending payload: \x30\x0D\x0A\x0D\x0A
>>> homekit_client_process: [Client 1] Finished processing
>>> HomeKit: [Client 1] Closing client connection from 192.168.178.28

Unfortunally I did not got the wolf_test_task(); to work, or maybe I don't know how? What I did is adding it to my main.c code, it compiled but did not give any output?

If you need any specifics, please let me know, I'm happy to follow your steps.

@gojimmypi
Copy link
Contributor

@AchimPieters that's curious the SRP is failing on your ESP32-C2, but not the ESP32. Do you have any other flavors to test with?

Can you provide your stack and IRAM settings? Here are mine:

IRAM

image
image

Stack

image

Heap

image

See also my:

In particular, although the ESP8266 is configured for #define FP_MAX_BITS MIN_FFDHE_FP_MAX_BITS due to limited memory, the C2 is using the full-sized #define FP_MAX_BITS (8192 * 2).

I think we tried this before, but what happens if you disabled all hardware acceleration? (There's limited capability in the ESP32-C2 anyhow)

To disabled all HW acceleration:

    #define NO_ESP32_CRYPT
    #define NO_WOLFSSL_ESP32_CRYPT_HASH
    #define NO_WOLFSSL_ESP32_CRYPT_AES
    #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI

Let's see what the results of the wolfssl_test are on your device. Would you please build and flash that onto your device and let me know the results?

For completeness, here's my ESP32-C2 log showing a passing SRP test:

ESP-ROM:esp8684-api2-20220127
Build:Jan 27 2022
rst:0x1 (POWERON),boot:0xc (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5c80,len:0x16f0
load:0x403acb70,len:0xbb0
load:0x403aeb70,len:0x27e0
entry 0x403acb7a
I (21) boot: ESP-IDF v5.2-dev-3903-g66992aca7a-dirty 2nd stage bootloader
I (21) boot: compile time Apr  5 2024 14:59:47
I (22) boot: chip revision: v1.0
I (26) boot.esp32c2: MMU Page Size  : 32K
I (31) boot.esp32c2: SPI Speed      : 60MHz
I (36) boot.esp32c2: SPI Mode       : DIO
I (40) boot.esp32c2: SPI Flash Size : 2MB
I (45) boot: Enabling RNG early entropy source...
I (50) boot: Partition Table:
I (54) boot: ## Label            Usage          Type ST Offset   Length
I (61) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (69) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (76) boot:  2 factory          factory app      00 00 00010000 00177000
I (84) boot: End of partition table
I (88) esp_image: segment 0: paddr=00010020 vaddr=3c0a8020 size=2cf58h (184152)map
I (138) esp_image: segment 1: paddr=0003cf80 vaddr=3fcaa4e0 size=015bch (  5564 load
I (139) esp_image: segment 2: paddr=0003e544 vaddr=40380000 size=01ad4h (  6868 load
I (145) esp_image: segment 3: paddr=00040020 vaddr=42000020 size=a7570h (685424 map
I (305) esp_image: segment 4: paddr=000e7598 vaddr=40381ad4 size=08a00h ( 35328 load
I (318) boot: Loaded app from partition at offset 0x10000
I (318) boot: Disabling RNG early entropy source...
I (329) cpu_start: Unicore app
I (338) cpu_start: Pro cpu start user code
I (338) cpu_start: cpu freq: 120000000 Hz
I (338) cpu_start: Application information:
I (341) cpu_start: Project name:     wolfssl_test
I (346) cpu_start: App version:      v5.6.6-stable-2389-g6f9363a31-d
I (353) cpu_start: Compile time:     Apr  5 2024 14:58:51
I (359) cpu_start: ELF file SHA256:  cc79336b1...
I (365) cpu_start: ESP-IDF:          v5.2-dev-3903-g66992aca7a-dirty
I (372) cpu_start: Min chip rev:     v1.0
I (377) cpu_start: Max chip rev:     v1.99
I (381) cpu_start: Chip rev:         v1.0
I (386) heap_init: Initializing. RAM available for dynamic allocation:
I (393) heap_init: At 3FCACDD0 len 0002FDA0 (191 KiB): RAM
I (399) heap_init: At 3FCDCB70 len 0000294C (10 KiB): RAM
I (407) spi_flash: detected chip: generic
I (410) spi_flash: flash io: dio
I (414) sleep: Configure to isolate all GPIO pins in sleep state
I (421) sleep: Enable automatic switching of GPIO sleep configuration
I (428) I (440) wolfssl_test: ------------------ wolfSSL Test Example ---------------
I (440) wolfssl_test: --------------------------------------------------------
I (445) wolfssl_test: --------------------------------------------------------
I (453) wolfssl_test: ---------------------- BEGIN MAIN ----------------------
I (460) wolfssl_test: --------------------------------------------------------
I (468) wolfssl_test: --------------------------------------------------------
I (476) wolfssl_test: Stack Start: 0x3fcb1680
W (481) wolfssl_test: Found WOLFSSL_ESP_NO_WATCHDOG, disabling...
W (488) esp32_util: No known rtc_wdt_protect_off for this platform.
I (495) wolfssl_test: CONFIG_ESP_MAIN_TASK_STACK_SIZE = 10500 bytes (2625 words
I (504) wolfssl_test: Stack Start HWM: 9296 bytes
I (508) esp32_util: Extended Version and Platform Information.
I (515) esp32_util: Chip revision: v1.0
I (519) esp32_util: SSID and plain text WiFi password not displayed in startup ogs.
I (528) esp32_util:   Define SHOW_SSID_AND_PASSWORD to enable display.
I (535) esp32_util: Using wolfSSL user_settings.h in //mnt//c//workspace//wolfsl-gojimmypi//IDE//Espressif//ESP-IDF//Examples//wolfssl_test//components//wolfsl//include//user_settings.h
I (552) esp32_util: LIBWOLFSSL_VERSION_STRING = 5.7.0
I (558) esp32_util: LIBWOLFSSL_VERSION_HEX = 5007000
I (564) esp32_util: CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ = 120 MHz
I (570) esp32_util: Stack HWM: 9296
I (574) esp32_util:
I (577) esp32_util: Macro Name                 Defined   Not Defined
I (584) esp32_util: ------------------------- --------- -------------
I (591) esp32_util: NO_ESPIDF_DEFAULT........                 X
I (598) esp32_util: HW_MATH_ENABLED..........                 X
I (605) esp32_util: WOLFSSL_SHA224...........     X
I (610) esp32_util: WOLFSSL_SHA384...........     X
I (616) esp32_util: WOLFSSL_SHA512...........     X
I (621) esp32_util: WOLFSSL_SHA3.............     X
I (627) esp32_util: HAVE_ED25519.............     X
I (632) esp32_util: HAVE_AES_ECB.............                 X
I (639) esp32_util: HAVE_AES_DIRECT..........                 X
I (646) esp32_util: USE_FAST_MATH............     X
I (651) esp32_util: WOLFSSL_SP_MATH_ALL......                 X
I (658) esp32_util: SP_MATH..................                 X
I (664) esp32_util: WOLFSSL_HW_METRICS.......     X
I (670) esp32_util: RSA_LOW_MEM..............     X
I (675) esp32_util: SMALL_SESSION_CACHE......                 X
I (682) esp32_util: WC_NO_HARDEN.............                 X
I (689) esp32_util: TFM_TIMING_RESISTANT.....     X
I (694) esp32_util: ECC_TIMING_RESISTANT.....     X
I (700) esp32_util: WC_NO_CACHE_RESISTANT....     X
I (705) esp32_util: WC_AES_BITSLICED.........                 X
I (712) esp32_util: WOLFSSL_AES_NO_UNROLL....                 X
I (719) esp32_util: TFM_TIMING_RESISTANT.....     X
I (724) esp32_util: ECC_TIMING_RESISTANT.....     X
I (730) esp32_util: WC_RSA_BLINDING..........     X
I (735) esp32_util: NO_WRITEV................     X
I (741) esp32_util: FREERTOS.................     X
I (746) esp32_util: NO_WOLFSSL_DIR...........     X
I (752) esp32_util: WOLFSSL_NO_CURRDIR.......     X
I (757) esp32_util: WOLFSSL_LWIP.............     X
I (763) esp32_util:
I (766) esp32_util: Compiler Optimization: Default
I (771) esp32_util:
I (774) esp32_util: LIBWOLFSSL_VERSION_GIT_ORIGIN = https://github.com/gojimmyp/wolfssl.git
I (783) esp32_util: LIBWOLFSSL_VERSION_GIT_BRANCH = ED25519_SHA2_fix
I (790) esp32_util: LIBWOLFSSL_VERSION_GIT_HASH = 6f9363a316b43b74471b4a7c2d94e3e3b4d0e74
I (799) esp32_util: LIBWOLFSSL_VERSION_GIT_SHORT_HASH = 6f9363a31
I (806) esp32_util: LIBWOLFSSL_VERSION_GIT_HASH_DATE = 'Fri Apr 5 10:25:22 2024-0700'
I (815) esp32_util: CONFIG_IDF_TARGET = esp32c2
I (820) esp32_util: Found WOLFSSL_ESP_NO_WATCHDOG
I (825) esp32_util: CONFIG_MAIN_TASK_STACK_SIZE: 10500
I (831) esp32_util: CONFIG_ESP_MAIN_TASK_STACK_SIZE: 10500
I (837) esp32_util: CONFIG_TIMER_TASK_STACK_SIZE: 3584
I (843) esp32_util: CONFIG_TIMER_TASK_STACK_DEPTH: 2048
I (849) esp32_util: Stack HWM: 9112
I (853) esp32_util: ESP32_CRYPT is enabled for ESP32-C2.
I (859) esp32_util: NO_WOLFSSL_ESP32_CRYPT_AES is defined! (disabled HW AES).
I (867) esp32_util: NO_WOLFSSL_ESP32_CRYPT_RSA_PRI defined! (disabled HW RSA)
I (875) esp32_util: NOT SINGLE_THREADED
I (879) esp32_util: Boot count: 1
I (883) wolfssl_test: Stack HWM: 9112

I (888) wc_port: wc_InitMutex success heap = 182080
------------------------------------------------------------------------------
 wolfSSL version 5.7.0
------------------------------------------------------------------------------
I (909) wc_test: Here we go with srp_test!
I (914) wc_port: wc_InitMutex success heap = 163748
SRP      test passed!
error    test passed!
MEMORY   test passed!
base64   test passed!
base16   test passed!
AES      test passed!
SHA-224  test passed!
HMAC-SHA224 test passed!
AES192   test passed!
SHA-256  test passed!
SHA-224  test passed!
SHA      test passed!
ED25519  test passed!
SHA-224  test passed!
SHA-384  test passed!
SHA-512  test passed!
SHA-512/224  test passed!
SHA-512/256  test passed!
etc...

I expect the SRP test to pass on your device. We can then compare if there are any configs in the user_settings.h and/or sdkconfig,defaults that are different from yours. It is possible that something else is going on, and the current SRP test is missing something that the Apple Homekit is doing.

@AchimPieters
Copy link
Author

AchimPieters commented Apr 8, 2024

@gojimmypi Can you provide your stack and IRAM settings? Sure, here are mine:

ESP32C2 IRAM:

# CONFIG_GPIO_CTRL_FUNC_IN_IRAM is not set

CONFIG_GPTIMER_ISR_HANDLER_IN_IRAM=y
# CONFIG_GPTIMER_CTRL_FUNC_IN_IRAM is not set
# CONFIG_GPTIMER_ISR_IRAM_SAFE is not set

# CONFIG_I2C_ISR_IRAM_SAFE is not set

# CONFIG_LEDC_CTRL_FUNC_IN_IRAM is not set

# CONFIG_SPI_MASTER_IN_IRAM is not set
CONFIG_SPI_MASTER_ISR_IN_IRAM=y
# CONFIG_SPI_SLAVE_IN_IRAM is not set
CONFIG_SPI_SLAVE_ISR_IN_IRAM=y

# CONFIG_UART_ISR_IN_IRAM is not set

CONFIG_ESP_EVENT_POST_FROM_IRAM_ISR=y

CONFIG_PERIPH_CTRL_FUNC_IN_IRAM=y

CONFIG_GDMA_CTRL_FUNC_IN_IRAM=y

CONFIG_ESP_SPI_BUS_LOCK_ISR_FUNCS_IN_IRAM=y

# CONFIG_ESP_PANIC_HANDLER_IRAM is not set

CONFIG_ESP_WIFI_IRAM_OPT=y
# CONFIG_ESP_WIFI_EXTRA_IRAM_OPT is not set
CONFIG_ESP_WIFI_RX_IRAM_OPT=y

# CONFIG_ESP_WIFI_SLP_IRAM_OPT is not set

CONFIG_HAL_SPI_MASTER_FUNC_IN_IRAM=y
CONFIG_HAL_SPI_SLAVE_FUNC_IN_IRAM=y

# CONFIG_LWIP_IRAM_OPTIMIZATION is not set
# CONFIG_LWIP_EXTRA_IRAM_OPTIMIZATION is not set

CONFIG_POST_EVENTS_FROM_IRAM_ISR=y

CONFIG_ESP32_WIFI_IRAM_OPT=y
CONFIG_ESP32_WIFI_RX_IRAM_OPT=y

ESP32C2 STACK:

CONFIG_COMPILER_STACK_CHECK_MODE_NONE=y
# CONFIG_COMPILER_STACK_CHECK_MODE_NORM is not set
# CONFIG_COMPILER_STACK_CHECK_MODE_STRONG is not set
# CONFIG_COMPILER_STACK_CHECK_MODE_ALL is not set

CONFIG_ESP_SYSTEM_EVENT_TASK_STACK_SIZE=2304
CONFIG_ESP_MAIN_TASK_STACK_SIZE=3584

CONFIG_ESP_MINIMAL_SHARED_STACK_SIZE=2048

CONFIG_ESP_SYSTEM_HW_STACK_GUARD=y

CONFIG_ESP_IPC_TASK_STACK_SIZE=1024

CONFIG_ESP_TIMER_TASK_STACK_SIZE=3584

# CONFIG_FATFS_LFN_STACK is not set

# CONFIG_FREERTOS_CHECK_STACKOVERFLOW_NONE is not set
# CONFIG_FREERTOS_CHECK_STACKOVERFLOW_PTRVAL is not set
CONFIG_FREERTOS_CHECK_STACKOVERFLOW_CANARY=y

CONFIG_FREERTOS_IDLE_TASK_STACKSIZE=1536

CONFIG_FREERTOS_TIMER_TASK_STACK_DEPTH=2048

# CONFIG_FREERTOS_WATCHPOINT_END_OF_STACK is not set

CONFIG_FREERTOS_ISR_STACKSIZE=1536

CONFIG_LWIP_TCPIP_TASK_STACK_SIZE=3072

CONFIG_PTHREAD_TASK_STACK_SIZE_DEFAULT=3072
CONFIG_PTHREAD_STACK_MIN=768

CONFIG_MDNS_TASK_STACK_SIZE=4096

CONFIG_STACK_CHECK_NONE=y
# CONFIG_STACK_CHECK_NORM is not set
# CONFIG_STACK_CHECK_STRONG is not set
# CONFIG_STACK_CHECK_ALL is not set

CONFIG_SYSTEM_EVENT_TASK_STACK_SIZE=2304
CONFIG_MAIN_TASK_STACK_SIZE=3584

CONFIG_IPC_TASK_STACK_SIZE=1024
CONFIG_TIMER_TASK_STACK_SIZE=3584

CONFIG_TIMER_TASK_STACK_DEPTH=2048

CONFIG_TCPIP_TASK_STACK_SIZE=3072

CONFIG_ESP32_PTHREAD_TASK_STACK_SIZE_DEFAULT=3072
CONFIG_ESP32_PTHREAD_STACK_MIN=768

ESP32C2 HEAP:

CONFIG_ESP_ROM_HAS_HEAP_TLSF=y
# CONFIG_FATFS_LFN_HEAP is not set

#
# Heap memory debugging
#
CONFIG_HEAP_POISONING_DISABLED=y
# CONFIG_HEAP_POISONING_LIGHT is not set
# CONFIG_HEAP_POISONING_COMPREHENSIVE is not set
CONFIG_HEAP_TRACING_OFF=y
# CONFIG_HEAP_TRACING_STANDALONE is not set
# CONFIG_HEAP_TRACING_TOHOST is not set
# CONFIG_HEAP_USE_HOOKS is not set
# CONFIG_HEAP_TASK_TRACKING is not set
# CONFIG_HEAP_ABORT_WHEN_ALLOCATION_FAILS is not set
CONFIG_HEAP_TLSF_USE_ROM_IMPL=y
CONFIG_HEAP_TLSF_CHECK_PATCH=y
# end of Heap memory debugging

regrading: disabled all HW acceleration:

    #define NO_ESP32_CRYPT
    #define NO_WOLFSSL_ESP32_CRYPT_HASH
    #define NO_WOLFSSL_ESP32_CRYPT_AES
    #define NO_WOLFSSL_ESP32_CRYPT_RSA_PRI

they are disabled already?

@AchimPieters
Copy link
Author

AchimPieters commented Apr 8, 2024

@gojimmypi the results of the wolfssl_test are as following:

ESP32

docker run -it -v ~/esp32-homekit-demo-gojimmypi:/project -w /project espressif/idf:latest

cd ./esp32-homekit-demo-gojimmypi

cd /wolfssl/IDE/Espressif/ESP-IDF/examples/wolfssl_test

idf.py set-target esp32

idf.py build

esptool.py erase_flash

python -m esptool --chip esp32 -b 460800 --before default_reset --after hard_reset write_flash --flash_mode dio --flash_size 2MB --flash_freq 40m 0x1000 build/bootloader/bootloader.bin 0x8000 build/partition_table/partition-table.bin 0x10000 build/wolfssl_test.bin

Here are the results:
ESP32 LOG.txt

ESP32C2

docker run -it -v ~/esp32-homekit-demo-gojimmypi:/project -w /project espressif/idf:latest

cd ./esp32-homekit-demo-gojimmypi

cd /wolfssl/IDE/Espressif/ESP-IDF/examples/wolfssl_test

idf.py set-target esp32c2

idf.py build

esptool.py erase_flash

python -m esptool --chip esp32c2 -b 460800 --before default_reset --after hard_reset write_flash --flash_mode dio --flash_size 2MB --flash_freq 40m 0x1000 build/bootloader/bootloader.bin 0x8000 build/partition_table/partition-table.bin 0x10000 build/wolfssl_test.bin

Here are the results:
ESP32C2 LOG.txt

So, I think I did everything you asked...

@gojimmypi
Copy link
Contributor

here are mine: ... ESP32C2 STACK:

aha! The first thing I notice is our sdkconfig settings differences, in particular the main task stack:

CONFIG_ESP_MAIN_TASK_STACK_SIZE=10500 vs CONFIG_ESP_MAIN_TASK_STACK_SIZE=3584

That alone is not necessarily the root cause, as I see you have several freeRTOS tasks, such as the one in led/main/main.c allocating only 2K stack.

See the Espressif docs on xTaskCreate.

There's also the components/homekit/src/server.c that appears to allocate only 1664 bytes if ESP_IDF is not defined.

I don't recognize the ESP_IDF macro. It is not defined in my wolfssl_test project, nor could I find one in any of the source files.

There's a cmake definition in the esp32-button, and the homekit component but I'm not certain that the task calling the wolfSSL SRP sees that & the appropriate stack size.

It might be a good idea to ensure we know the exact value of SERVER_TASK_STACK during the call. I would think that it is always the 12288 value, as you indicated the code works for the ESP32. Still, I'd try to set it to an known value and put in a ESP_LOGI() to confirm.

It is challenging for me to not have any of the Apple Homekit products. Are you aware of any emulation environments that I might be able to use to test? It would make this exercise much easier.

You might also consider turning on stack smashing features to see if the code will tell us if indeed there's a stack problem. See the CONFIG_STACK_CHECK_NONE setting.

Here are the results: ESP32C2 LOG.txt

Excellent! That at least confirms the SRP test is passing on your ESP32C2. Even more evidence this is a stack setting issue.

@AchimPieters
Copy link
Author

AchimPieters commented Apr 9, 2024

@gojimmypi good to hear this gives more insight in the problem, regarding your question about an emulator, there are a few out there depending on which you have a good feeling, there is this one for example: https://dev.to/ianito/how-to-emulate-ios-on-linux-with-docker-4gj3

or

https://hackintosh.com

and then https://developer.apple.com/documentation/homekit/testing_your_app_with_the_homekit_accessory_simulator

I have an old iPhone, that I could send to you by mail?

I'm going to look into the intel you provided...

@gojimmypi
Copy link
Contributor

Hi @AchimPieters - I know you mentioned you've not been feeling well... so no rush here.. just wondering if you have an update on the ESP32-C2 SRP Homekit failure that you were seeing?

I took a brief look at some of the Apple emulators and simulators. One cannot deny the simplicity of testing on real hardware. If you are still seeing problems & have a loaner iPhone that I can for testing, that would be great! That's a very generous offer. Please reach out to me via email: jim (at) wolfssl.com

In return I can probably arrange to send you some wolfssl swag & I'll of course return the device upon completion of our exercise.

@AchimPieters
Copy link
Author

@gojimmypi no update yet, I'm going to try doing some test this weekend ;) I'm currently a little behind, but I'll be fine!

I will email you, to discuss some things regarding the iPhone.

@AchimPieters
Copy link
Author

@gojimmypi

Okay, let's start from my repro at this moment: https://github.com/AchimPieters/esp32-homekit-demo with my current user_settings.h and go from here...

@gojimmypi
Copy link
Contributor

@AchimPieters - excellent! I believe I have all the issues worked out.

Having the iPhone was quite helpful. Thank you again.

See your PR AchimPieters/esp32-homekit-demo#3 and wolfSSL #7505.

For reference, here are my instructions to test on my fork:

# set TEST_DIR to fully qualified path:
# export TEST_DIR=~/yourdir
export TEST_DIR=/mnt/c/test-homekit

cd $TEST_DIR

mkdir test-homekit
cd    test-homekit

# get a copy of the gojimmypi fork of wolfSSL (see PR)
git clone https://github.com/gojimmypi/wolfssl.git wolfssl-gojimmypi
cd wolfssl-gojimmypi
git checkout PR-Apple-Homekit-SRP-fix
cd ..

# if wolfSSL was fetched to `wolfssl` or 'wolfssl-$USER` or `wolfssl-master`
# then cmake would find it if in parent directory of example
# since it is not, we set the WOLFSSL_ROOT source code directory an environment variable:

export WOLFSSL_ROOT=$TEST_DIR/wolfssl-gojimmypi

# get a copy of the gojimmypi esp32-homekit-demo fork (see PR)
git clone --recursive https://github.com/gojimmypi/esp32-homekit-demo.git esp32-homekit-demo-gojimmypi
cd  ./esp32-homekit-demo-gojimmypi
git remote add upstream https://github.com/AchimPieters/esp32-homekit-demo.git
git checkout PR-wolfssl

# Windows users may need:
git config core.fileMode false

cd ./examples/led

# Set path for your ESP-IDF, shown here for v5.2 in WSL for VisualGDB
WRK_IDF_PATH=/mnt/c/SysGCC/esp32/esp-idf/v5.2

. ${WRK_IDF_PATH}/export.sh

idf.py set-target esp32c2
idf.py menuconfig
idf.py build

@AchimPieters
Copy link
Author

@gojimmypi

That's fantastic news! It's great to see progress being made. However, despite following all your instructions, I wasn't able to reproduce it. Even after accepting your merge request and trying again, I still couldn't get it to work. I suspect the difference in operating systems, Windows versus macOS, might be causing the issue.

I had to revert the merge again. Additionally, the changes to the local, static, stale copy of wolfSSL are causing complications, along with the fact that I'm using Docker.

I'd prefer to use wolfSSL via https://components.espressif.com/ because I've built this repro for beginners, so they can start with creating a HomeKit device in a simple and educational way. For this reason, I'd like to offer a repro that includes all the necessary components to start without any extra steps.

Once again, it's amazing that it worked for you! Now, I'll need to figure out how to make it work on my platform.

@gojimmypi
Copy link
Contributor

@AchimPieters

That's fantastic news! It's great to see progress being made.

Yes! I'm confident we'll be able to get this working properly.

after accepting your merge request and trying again, I still couldn't get it to work

I'll need a little more than "couldn't get it to work", please. New error messages? Some of the changes to your code in my repo are essential to getting the ESP2-C2 to work, as well as changes to wolfSSL: Key to this working is also the merge of #7505 into wolfSSL, which has not occurred yet.

You can continue to test with my branch, if you'd like. Any other option is unlikely to succeed without code changes.

the changes to the local, static, stale copy of wolfSSL are causing complications.

I'll need specifics to "complications" in order to be able to help.

There must be changes to wolfSSL in order for the ESP32-C2 to work properly for all your Homekit examples and all the Espressif targets. I'm particularly interested in the value of WOLFSSL_ROOT.

I suspect the difference in operating systems, Windows versus macOS, might be causing the issue

I'm using the Espressif v5.2. If you are using the same version, I think it is highly unlikely (though not impossible) that OS difference would cause problems. I need to see the errors you are encountering.

I'd prefer to use wolfSSL via https://components.espressif.com/

Yes, I agree, however that's not going to happen today, nor in the immediate future, at least not for the official wolfSSL. I do however, have a staging instance of wolfssl called mywolfssl that could be used for preliminary testing of that. But I would need to publish an update, and I'd prefer to do that from the official wolfSSL branch after my PR 7505 there gets merged. I want to make sure this is working for you before updating those published libraries.

Using the managed component would also address any possible license issues of distributing wolfSSL source code with your project or anyone else that uses it and distributes it.

I suggest trying my PR-wolfssl branch of your repo, even making wolfSSL a temporary, static local copy with my wolfSSL PR Branch called PR-Apple-Homekit-SRP-fix. Baby steps. Let's get that working for you and go from there.

I assumed you'd do the testing as noted in AchimPieters/esp32-homekit-demo#3 instruction before merging. The instructions there are for the forks and branches.

The most important thing to look for is the value of WOLFSSL_ROOT. I wonder if that's being set and possibly overridden in your docker environment? It is important, and I display it during the processing of the wolfSSL cmake file. Look for instances of WOLFSSL_ROOT in the cmake output. The most important one being the value. For instance for me:

-- WOLFSSL_EXTRA_PROJECT_DIR=C:/workspace/wolfssl-gojimmypi/src/
-- WOLFSSL_FOUND_IDF=0
-- WOLFSSL_PROJECT_DIR=C:/workspace/wolfssl-gojimmypi/IDE/Espressif/ESP-IDF/examples/wolfssl_client/components/wolfssl
-- WOLFSSL_ROOT=C:/workspace/wolfssl-gojimmypi

We need to also ensure you are not using wolfSSL in the ESP-IDF components. (see WOLFSSL_FOUND_IDF=0, above). The WOLFSSL_ROOT path should address that.

When following those instructions, I'd like to see the value of WOLFSSL_ROOT environment variable from this step, and proof that the path exists and contains my latest update to wolfSSL:

export WOLFSSL_ROOT=$TEST_DIR/wolfssl-gojimmypi

I'd like to see full build/flash logs and any relevant device output on the serial monitor, please. I realize that we're 9 time zones apart and it is later for you at the moment, but if you can get that to me today, I'll work on it today.

Thanks so much for your persistence. We're really close, and I am completely confident we'll have a solid solution soon.

@AchimPieters
Copy link
Author

@gojimmypi Her is what I tried so far:

Followed your instructions, as far as the are applicable on my system (macOS), I made a shell script (.sh) file and run it:

# set TEST_DIR to fully qualified path:
# export TEST_DIR=~/yourdir
export TEST_DIR=/mnt/c/test-homekit

cd $TEST_DIR

mkdir test-homekit
cd    test-homekit

# get a copy of the gojimmypi fork of wolfSSL (see PR)
git clone https://github.com/gojimmypi/wolfssl.git wolfssl-gojimmypi
cd wolfssl-gojimmypi
git checkout PR-Apple-Homekit-SRP-fix
cd ..

# if wolfSSL was fetched to `wolfssl` or 'wolfssl-$USER` or `wolfssl-master`
# then cmake would find it if in parent directory of example
# since it is not, we set the WOLFSSL_ROOT source code directory an environment variable:

export WOLFSSL_ROOT=$TEST_DIR/wolfssl-gojimmypi

# get a copy of the gojimmypi esp32-homekit-demo fork (see PR)
git clone --recursive https://github.com/gojimmypi/esp32-homekit-demo.git esp32-homekit-demo-gojimmypi
cd  ./esp32-homekit-demo-gojimmypi
git remote add upstream https://github.com/AchimPieters/esp32-homekit-demo.git
git checkout PR-wolfssl

with this as result:

Last login: Sun May  5 17:16:26 on ttys000
achimpieters@Achims-iMac ~ % bash run.sh
run.sh: line 5: cd: /mnt/c/test-homekit: No such file or directory
Cloning into 'wolfssl-gojimmypi'...
remote: Enumerating objects: 184809, done.
remote: Counting objects: 100% (2598/2598), done.
remote: Compressing objects: 100% (1188/1188), done.
remote: Total 184809 (delta 1632), reused 2132 (delta 1323), pack-reused 182211
Receiving objects: 100% (184809/184809), 628.44 MiB | 21.62 MiB/s, done.
Resolving deltas: 100% (135634/135634), done.
Updating files: 100% (2730/2730), done.
branch 'PR-Apple-Homekit-SRP-fix' set up to track 'origin/PR-Apple-Homekit-SRP-fix'.
Switched to a new branch 'PR-Apple-Homekit-SRP-fix'
Cloning into 'esp32-homekit-demo-gojimmypi'...
remote: Enumerating objects: 4829, done.
remote: Counting objects: 100% (463/463), done.
remote: Compressing objects: 100% (316/316), done.
remote: Total 4829 (delta 212), reused 238 (delta 143), pack-reused 4366
Receiving objects: 100% (4829/4829), 40.91 MiB | 18.90 MiB/s, done.
Resolving deltas: 100% (1891/1891), done.
Submodule 'components/esp32-button' (https://[email protected]/AchimPieters/esp32-button.git) registered for path 'components/esp32-button'
Submodule 'components/esp32-library' (https://[email protected]/UncleRus/esp-idf-lib.git) registered for path 'components/esp32-library'
Submodule 'components/homekit' (https://[email protected]/AchimPieters/esp32-homekit.git) registered for path 'components/homekit'
Cloning into '/Users/achimpieters/test-homekit/esp32-homekit-demo-gojimmypi/components/esp32-button'...
remote: Enumerating objects: 45, done.        
remote: Counting objects: 100% (45/45), done.        
remote: Compressing objects: 100% (41/41), done.        
remote: Total 45 (delta 18), reused 19 (delta 3), pack-reused 0        
Receiving objects: 100% (45/45), 105.84 KiB | 1.86 MiB/s, done.
Resolving deltas: 100% (18/18), done.
Cloning into '/Users/achimpieters/test-homekit/esp32-homekit-demo-gojimmypi/components/esp32-library'...
remote: Enumerating objects: 13128, done.        
remote: Counting objects: 100% (1783/1783), done.        
remote: Compressing objects: 100% (327/327), done.        
remote: Total 13128 (delta 1513), reused 1555 (delta 1447), pack-reused 11345        
Receiving objects: 100% (13128/13128), 4.70 MiB | 17.29 MiB/s, done.
Resolving deltas: 100% (9043/9043), done.
Cloning into '/Users/achimpieters/test-homekit/esp32-homekit-demo-gojimmypi/components/homekit'...
remote: Enumerating objects: 242, done.        
remote: Counting objects: 100% (242/242), done.        
remote: Compressing objects: 100% (156/156), done.        
remote: Total 242 (delta 139), reused 142 (delta 79), pack-reused 0        
Receiving objects: 100% (242/242), 344.93 KiB | 4.48 MiB/s, done.
Resolving deltas: 100% (139/139), done.
Submodule path 'components/esp32-button': checked out 'a5308eb90a9232867e07eae829eb5fbff2960b99'
Submodule path 'components/esp32-library': checked out '47fabc5c770c460e9347969e7ade38d473a4355f'
Submodule path 'components/homekit': checked out 'e798825d7814a563b06f3da084f0114c3e415dd6'
branch 'PR-wolfssl' set up to track 'origin/PR-wolfssl'.
Switched to a new branch 'PR-wolfssl'
achimpieters@Achims-iMac ~ % 

Then I started my ESP-IDF environment (docker), Notice I run it from the main dir 'test-homekit', so all dirs are included!

achimpieters@Achims-iMac ~ % docker run -it -v ~/test-homekit:/project -w /project espressif/idf:latest

with this as the result

Last login: Sun May  5 17:17:38 on ttys000
achimpieters@Achims-iMac ~ % docker run -it -v ~/test-homekit:/project -w /project espressif/idf:latest
Detecting the Python interpreter
Checking "python3" ...
Python 3.10.12
"python3" has been detected
Checking Python compatibility
Checking other ESP-IDF version.
Adding ESP-IDF tools to PATH...
Checking if Python packages are up to date...
Requirement files:
 - /opt/esp/idf/tools/requirements/requirements.core.txt
Python being checked: /opt/esp/python_env/idf5.4_py3.10_env/bin/python
Python requirements are satisfied.
Added the following directories to PATH:
  /opt/esp/idf/components/espcoredump
  /opt/esp/idf/components/partition_table
  /opt/esp/idf/components/app_update
  /opt/esp/tools/xtensa-esp-elf-gdb/14.2_20240403/xtensa-esp-elf-gdb/bin
  /opt/esp/tools/riscv32-esp-elf-gdb/14.2_20240403/riscv32-esp-elf-gdb/bin
  /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin
  /opt/esp/tools/riscv32-esp-elf/esp-13.2.0_20240305/riscv32-esp-elf/bin
  /opt/esp/tools/esp32ulp-elf/2.38_20240113/esp32ulp-elf/bin
  /opt/esp/tools/cmake/3.24.0/bin
  /opt/esp/tools/openocd-esp32/v0.12.0-esp32-20240318/openocd-esp32/bin
  /opt/esp/tools/qemu-xtensa/esp_develop_8.2.0_20240122/qemu/bin
  /opt/esp/tools/qemu-riscv32/esp_develop_8.2.0_20240122/qemu/bin
  /opt/esp/tools/xtensa-esp-elf-gdb/14.2_20240403/xtensa-esp-elf-gdb/bin
  /opt/esp/tools/riscv32-esp-elf-gdb/14.2_20240403/riscv32-esp-elf-gdb/bin
  /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin
  /opt/esp/tools/riscv32-esp-elf/esp-13.2.0_20240305/riscv32-esp-elf/bin
  /opt/esp/tools/esp32ulp-elf/2.38_20240113/esp32ulp-elf/bin
  /opt/esp/tools/cmake/3.24.0/bin
  /opt/esp/tools/openocd-esp32/v0.12.0-esp32-20240318/openocd-esp32/bin
  /opt/esp/tools/qemu-xtensa/esp_develop_8.2.0_20240122/qemu/bin
  /opt/esp/tools/qemu-riscv32/esp_develop_8.2.0_20240122/qemu/bin
  /opt/esp/python_env/idf5.4_py3.10_env/bin
  /opt/esp/idf/tools
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

root@7ee758f7e4a3:/project# cd ./examples/led
bash: cd: ./examples/led: No such file or directory
root@7ee758f7e4a3:/project# ls
esp32-homekit-demo-gojimmypi  wolfssl-gojimmypi
root@7ee758f7e4a3:/project# cd esp32-homekit-demo-gojimmypi
root@7ee758f7e4a3:/project/esp32-homekit-demo-gojimmypi# 

root@7ee758f7e4a3:/project/esp32-homekit-demo-gojimmypi# cd ./examples/led
root@7ee758f7e4a3:/project/esp32-homekit-demo-gojimmypi/examples/led# idf.py set-target esp32
Adding "set-target"'s dependency "fullclean" to list of commands with default set of options.
Executing action: fullclean
Build directory '/project/esp32-homekit-demo-gojimmypi/examples/led/build' not found. Nothing to clean.
Executing action: set-target
Set Target to: esp32, new sdkconfig will be created.
Running cmake in directory /project/esp32-homekit-demo-gojimmypi/examples/led/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/opt/esp/python_env/idf5.4_py3.10_env/bin/python -DESP_PLATFORM=1 -DIDF_TARGET=esp32 -DCCACHE_ENABLE=1 /project/esp32-homekit-demo-gojimmypi/examples/led"...
-- Configuring for ESP32 variant: 
-- Partition table file: 
-- Found Git: /usr/bin/git (found version "2.34.1") 
-- Component directory /project/esp32-homekit-demo-gojimmypi/components/esp-idf does not contain a CMakeLists.txt file. No component will be added
-- Component directory /project/esp32-homekit-demo-gojimmypi/components/esp32-library does not contain a CMakeLists.txt file. No component will be added
-- ccache will be used for faster recompilation
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin/xtensa-esp32-elf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building ESP-IDF components for target esp32
Dependencies lock doesn't exist, solving dependencies.
.Updating lock file at /project/esp32-homekit-demo-gojimmypi/examples/led/dependencies.lock
Processing 1 dependencies:
[1/1] idf (5.4.0)
-- WOLFSSL_ROOT = c:/workspace/wolfssl-gojimmypi
-- USERNAME = 
-- could not find USER or USERNAME
-- THIS_USER = 
-- wolfssl component CMAKE_BUILD_EARLY_EXPANSION:
-- Project sdkconfig file /project/esp32-homekit-demo-gojimmypi/examples/led/sdkconfig
Loading defaults file /project/esp32-homekit-demo-gojimmypi/examples/led/sdkconfig.defaults...
Loading defaults file /project/esp32-homekit-demo-gojimmypi/examples/led/sdkconfig.defaults.esp32...
-- Compiler supported targets: xtensa-esp-elf
-- Found Python3: /opt/esp/python_env/idf5.4_py3.10_env/bin/python (found version "3.10.12") found components: Interpreter 
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS - Success
-- App "main" version: 11fa20f
-- Adding linker script /project/esp32-homekit-demo-gojimmypi/examples/led/build/esp-idf/esp_system/ld/memory.ld
-- Adding linker script /project/esp32-homekit-demo-gojimmypi/examples/led/build/esp-idf/esp_system/ld/sections.ld.in
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.api.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Adding linker script /opt/esp/idf/components/soc/esp32/ld/esp32.peripherals.ld
-- WOLFSSL_ROOT = c:/workspace/wolfssl-gojimmypi
-- USERNAME = 
-- could not find USER or USERNAME
-- THIS_USER = 
-- ************************************************************************************************
-- wolfssl component config:
-- ************************************************************************************************
-- Starting FIND_WOLFSSL_DIRECTORY
CMake Error at /project/esp32-homekit-demo-gojimmypi/components/wolfssl/CMakeLists.txt:100 (message):
  WOLFSSL_ROOT Environment Variable not defined and wolfssl path not found in
  any parent.
Call Stack (most recent call first):
  /project/esp32-homekit-demo-gojimmypi/components/wolfssl/CMakeLists.txt:182 (FIND_WOLFSSL_DIRECTORY)


-- Configuring incomplete, errors occurred!
See also "/project/esp32-homekit-demo-gojimmypi/examples/led/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1, output of the command is in the /project/esp32-homekit-demo-gojimmypi/examples/led/build/log/idf_py_stderr_output_113 and /project/esp32-homekit-demo-gojimmypi/examples/led/build/log/idf_py_stdout_output_113
root@7ee758f7e4a3:/project/esp32-homekit-demo-gojimmypi/examples/led# idf.py menuconfig
Executing action: menuconfig
Running cmake in directory /project/esp32-homekit-demo-gojimmypi/examples/led/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=/opt/esp/python_env/idf5.4_py3.10_env/bin/python -DESP_PLATFORM=1 -DCCACHE_ENABLE=1 /project/esp32-homekit-demo-gojimmypi/examples/led"...
-- Configuring for ESP32 variant: 
-- Partition table file: 
-- IDF_TARGET is not set, guessed 'esp32' from sdkconfig '/project/esp32-homekit-demo-gojimmypi/examples/led/sdkconfig'
-- Found Git: /usr/bin/git (found version "2.34.1") 
-- Component directory /project/esp32-homekit-demo-gojimmypi/components/esp-idf does not contain a CMakeLists.txt file. No component will be added
-- Component directory /project/esp32-homekit-demo-gojimmypi/components/esp32-library does not contain a CMakeLists.txt file. No component will be added
-- ccache will be used for faster recompilation
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- The ASM compiler identification is GNU
-- Found assembler: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin/xtensa-esp32-elf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/esp/tools/xtensa-esp-elf/esp-13.2.0_20240305/xtensa-esp-elf/bin/xtensa-esp32-elf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building ESP-IDF components for target esp32
Processing 1 dependencies:
[1/1] idf (5.4.0)
-- WOLFSSL_ROOT = c:/workspace/wolfssl-gojimmypi
-- USERNAME = 
-- could not find USER or USERNAME
-- THIS_USER = 
-- wolfssl component CMAKE_BUILD_EARLY_EXPANSION:
-- Project sdkconfig file /project/esp32-homekit-demo-gojimmypi/examples/led/sdkconfig
Loading defaults file /project/esp32-homekit-demo-gojimmypi/examples/led/sdkconfig.defaults...
Loading defaults file /project/esp32-homekit-demo-gojimmypi/examples/led/sdkconfig.defaults.esp32...
-- Compiler supported targets: xtensa-esp-elf
-- Found Python3: /opt/esp/python_env/idf5.4_py3.10_env/bin/python (found version "3.10.12") found components: Interpreter 
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS
-- Performing Test C_COMPILER_SUPPORTS_WFORMAT_SIGNEDNESS - Success
-- App "main" version: 11fa20f
-- Adding linker script /project/esp32-homekit-demo-gojimmypi/examples/led/build/esp-idf/esp_system/ld/memory.ld
-- Adding linker script /project/esp32-homekit-demo-gojimmypi/examples/led/build/esp-idf/esp_system/ld/sections.ld.in
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.api.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.libgcc.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.newlib-data.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.syscalls.ld
-- Adding linker script /opt/esp/idf/components/esp_rom/esp32/ld/esp32.rom.newlib-funcs.ld
-- Adding linker script /opt/esp/idf/components/soc/esp32/ld/esp32.peripherals.ld
-- WOLFSSL_ROOT = c:/workspace/wolfssl-gojimmypi
-- USERNAME = 
-- could not find USER or USERNAME
-- THIS_USER = 
-- ************************************************************************************************
-- wolfssl component config:
-- ************************************************************************************************
-- Starting FIND_WOLFSSL_DIRECTORY
CMake Error at /project/esp32-homekit-demo-gojimmypi/components/wolfssl/CMakeLists.txt:100 (message):
  WOLFSSL_ROOT Environment Variable not defined and wolfssl path not found in
  any parent.
Call Stack (most recent call first):
  /project/esp32-homekit-demo-gojimmypi/components/wolfssl/CMakeLists.txt:182 (FIND_WOLFSSL_DIRECTORY)


-- Configuring incomplete, errors occurred!
See also "/project/esp32-homekit-demo-gojimmypi/examples/led/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1
root@7ee758f7e4a3:/project/esp32-homekit-demo-gojimmypi/examples/led# 

As you can see the error:

-- Starting FIND_WOLFSSL_DIRECTORY
CMake Error at /project/esp32-homekit-demo-gojimmypi/components/wolfssl/CMakeLists.txt:100 (message):
  WOLFSSL_ROOT Environment Variable not defined and wolfssl path not found in
  any parent.
Call Stack (most recent call first):
  /project/esp32-homekit-demo-gojimmypi/components/wolfssl/CMakeLists.txt:182 (FIND_WOLFSSL_DIRECTORY)

So I looked at 'set(WOLFSSL_ROOT "c:/workspace/wolfssl-gojimmypi")' You use 'c:/.....' That doesnt work on my mac and tried to change it to several paths but without anny result...

There has to be an easier way, as said here above 'because I've built this repro for beginners', Just say your reply, gonna read that now ;)

@gojimmypi
Copy link
Contributor

As you can see the error:

Aha! yes. Cool. I see the problem.

I'm so sorry. My bad. I thought I commented that out.

That certainly explains why it worked on my machine and not yours.

The local cmake assignment takes precedent over environment variables and searching parent directories for wolfSSL source:

set(WOLFSSL_ROOT "c:/workspace/wolfssl-gojimmypi")

Try setting that to the path of your local copy of my wolfssl branch PR-Apple-Homekit-SRP-fix for PR #7505.

Specifically edit line 33 of your esp32-homekit-demo/components/wolfssl/CMakeLists.txt with your path from the wolfssl-gojimmypi clone directory created with this:

git clone https://github.com/gojimmypi/wolfssl.git wolfssl-gojimmypi
cd wolfssl-gojimmypi
git checkout PR-Apple-Homekit-SRP-fix

You can either set a static value for our testing, or comment it out and use the environment variable. Once editing the file to comment it out, you might as well put in a static value for the wolfSSL location.

@gojimmypi
Copy link
Contributor

There has to be an easier way, as said here above 'because I've built this repro for beginners'

Oh, and as for that: yes, I completely agree. I had not expected you to merge the changes without testing. For an interim solution, I think setting the environment variable for the location of wolfSSL source code is fairly straightforward for beginners.

export WOLFSSL_ROOT=/path/to/wolfssl/

I've confirmed that simply commenting out the set(WOLFSSL_ROOT "c:/workspace/wolfssl-gojimmypi") also works (instead of setting a path in the cmake file) as long as wolfSSL and my fork of esp32-homekit-demo are both cloned from the same directory (but in your case, wolfssl not named with the -gojimmypi suffix).

The cmake file will automatically find wolfssl, wolfssl-$USER, or wolfSSL-master in the parent directories of an example.

I've updated my branch commenting out that path.

Once my wolfSSL PR #7505 gets merged and there's a new, official stable release (probably in a couple of months), the wofssl Managed Component is definitely the way to go, as you stated.

Let's get it confirmed working for you and then we can do some polishing.

@AchimPieters
Copy link
Author

AchimPieters commented May 5, 2024

@gojimmypi I have made progress!!

Here is what I did:

I took my working repro, then I cloned:

git clone https://github.com/gojimmypi/wolfssl.git wolfssl-gojimmypi
cd wolfssl-gojimmypi
git checkout PR-Apple-Homekit-SRP-fix

Replaced the directories, examples, scr, wolfcrypt and wolfssl in my 'wolfssl' directory under components. Then I changed the user_settings.h after the changes you made. I also changed the other files, as you did in your pull request.

Then I started compiling as always, I saw these warnings for the ESP32 module:

[778/935] Building C object esp-idf/wo...s/__idf_wolfssl.dir/src/ssl_load.c.obj
/project/components/wolfssl/src/ssl_load.c:56:10: warning: #warning ssl_bn.c does not need to be compiled separately from ssl.c [-Wcpp]
   56 |         #warning ssl_bn.c does not need to be compiled separately from ssl.c
      |          ^~~~~~~
[780/935] Building C object esp-idf/wo.../__idf_wolfssl.dir/src/ssl_p7p12.c.obj
/project/components/wolfssl/src/ssl_p7p12.c:37:10: warning: #warning ssl_p7p12.c does not need to be compiled separately from ssl.c [-Wcpp]
   37 |         #warning ssl_p7p12.c does not need to be compiled separately from ssl.c
      |          ^~~~~~~
[781/935] Building C object esp-idf/wo...s/__idf_wolfssl.dir/src/ssl_sess.c.obj
/project/components/wolfssl/src/ssl_sess.c:31:10: warning: #warning ssl_sess.c does not need to be compiled separately from ssl.c [-Wcpp]
   31 |         #warning ssl_sess.c does not need to be compiled separately from ssl.c
      |          ^~~~~~~

But it compiled and worked under HomeKit as usual. Then the test for the ESP32C2 and once again the warnings:

[704/859] Building C object esp-idf/wo...s/__idf_wolfssl.dir/src/ssl_load.c.obj
/project/components/wolfssl/src/ssl_load.c:56:10: warning: #warning ssl_bn.c does not need to be compiled separately from ssl.c [-Wcpp]
   56 |         #warning ssl_bn.c does not need to be compiled separately from ssl.c
      |          ^~~~~~~
[705/859] Building C object esp-idf/wo.../__idf_wolfssl.dir/src/ssl_p7p12.c.obj
/project/components/wolfssl/src/ssl_p7p12.c:37:10: warning: #warning ssl_p7p12.c does not need to be compiled separately from ssl.c [-Wcpp]
   37 |         #warning ssl_p7p12.c does not need to be compiled separately from ssl.c
      |          ^~~~~~~
[706/859] Building C object esp-idf/wo...s/__idf_wolfssl.dir/src/ssl_sess.c.obj
/project/components/wolfssl/src/ssl_sess.c:31:10: warning: #warning ssl_sess.c does not need to be compiled separately from ssl.c [-Wcpp]
   31 |         #warning ssl_sess.c does not need to be compiled separately from ssl.c
      |          ^~~~~~~
[858/859] Generating binary image from built executable

But it compiled and worked under HomeKit!! so progress!

Here is the update repro: https://github.com/AchimPieters/esp32-homekit-demo

@gojimmypi
Copy link
Contributor

Hi @AchimPieters !

it compiled and worked under HomeKit!!

YAY! That's excellent!!

Replaced the directories, examples, scr, wolfcrypt and wolfssl in my 'wolfssl' directory under components

oof, that's a bit of a brute-force method, but I'm glad it worked for you! <happy dance />

I saw these warnings for the ESP32 module [...]

Those warnings are because you didn't replace the CMakeLists.txt in your components/wolfssl directory with the latest version that was included in my PR. See in particular the set(COMPONENT_SRCEXCLUDE....

It's no big deal... it's just what it says: some files "do not need to be compiled separately from ssl.c".

I'll refresh my fork from your changes & create a new PR soon.

The important thing is you are using the changes in wolfSSL PR #7505 (in particular the changes to esp32_aes.c and esp32_sha.c).... and that those changes are working for you!! hooray! Once those changes are merged, I'll refresh the mywolfssl staging component and we can test with that. We are both in agreement that the Managed Components are by far the easiest to use for the beginner.

Thank you again for all your cooperation. I'm so glad it is working for you. Cheers!

@gojimmypi
Copy link
Contributor

Hello @AchimPieters -

I've created AchimPieters/esp32-homekit-demo#6 that adds Apple HomeKit support using wolfSSL as a Managed Component.

Pre-release version 5.7.1-Preview2e or later must be used.

See: https://components.espressif.com/components/wolfssl/wolfssl/versions/5.7.1-preview2e

I've only updated the LED example. I've included instructions in the README on adding this to other examples.

Basically just do this from the main directory:

idf.py add-dependency "wolfssl/wolfssl^5.7.1-preview2e"

and ensure the Enable Apple HomeKit options is checked with idf.py menuconfig, or as seen in in VisualGDB:

image

@gojimmypi
Copy link
Contributor

I'm closing this issue as I believe we have fixed this in #7505 and the latest Managed Component as described above..

Please feel free to open a new issue if anything else needs attention.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants