Experimental ESP-IDF v5 #810

llange · 2023-01-31T00:14:03Z

This PR is more or less ready for review.

It aims to bring compatibility with ESP-IDFv5, with the following caveats:

Same limitations as the ESP-IDFv4 branch on which it is based, see Experimental ESP-IDF v4 #806
Factory reset is absent (esp_flash_erase_region needs some work)
Mongoose is not (yet) ready to compile with TLS enabled.
There are still some warnings during compilation (mainly ADC which needs conversion + some others)
wolfSSL can't be (yet) compiled with OPENSSL defines (see Compilation error when defining OPENSSL_ALL or OPENSSL_EXTRA on ESP-IDFv5 : ssl.c:18011:22: error: size of array 'sha_test' is negative wolfSSL/wolfssl#6028)

Please note that:

wolfSSL has been updated to tag v5.5.4-stable, and needs a small patch to compile (see [Bug]: (sha512.c:1309:29: error: 'sha512' undeclared) wolfSSL/wolfssl#6024, [Bug]: esp32_sha.c:511:26: error: lvalue required as left operand of assignment wolfSSL/wolfssl#6026, ESP-IDFv5: Warning related to the uint32_t definition change wolfSSL/wolfssl#6029 fixed in master by ESP32 SHA384 fixes wolfSSL/wolfssl#6034)

diff --git a/wolfcrypt/src/port/Espressif/esp32_sha.c b/wolfcrypt/src/port/Espressif/esp32_sha.c
index 557bde8a9..0706fa7b1 100644
--- a/wolfcrypt/src/port/Espressif/esp32_sha.c
+++ b/wolfcrypt/src/port/Espressif/esp32_sha.c
@@ -507,8 +507,7 @@ int wc_esp_digest_state(WC_ESP32SHA* ctx, byte* hash)
 
     #if defined(WOLFSSL_SHA384)
         case SHA2_384:
-            SHA_LOAD_REG = SHA_384_LOAD_REG;
-            SHA_BUSY_REG = SHA_384_BUSY_REG;
+            DPORT_REG_WRITE(SHA_384_LOAD_REG, 1);
             break;
     #endif
 
@@ -551,7 +550,7 @@ int wc_esp_digest_state(WC_ESP32SHA* ctx, byte* hash)
      *    DPORT_SEQUENCE_REG_READ(address + i * 4);
      */
     esp_dport_access_read_buffer(
-        (word32*)(hash), /* the result will be found in hash upon exit     */
+        (uint32_t*)(hash), /* the result will be found in hash upon exit     */
         SHA_TEXT_BASE,   /* there's a fixed reg addy for all SHA           */
         wc_esp_sha_digest_size(ctx->sha_type) / sizeof(word32) /* # 4-byte */
     );
diff --git a/wolfcrypt/src/sha512.c b/wolfcrypt/src/sha512.c
index d42806976..deb60222d 100644
--- a/wolfcrypt/src/sha512.c
+++ b/wolfcrypt/src/sha512.c
@@ -1306,7 +1306,7 @@ static int InitSha384(wc_Sha384* sha384)
     sha384->ctx.isfirstblock = 1;
     if(sha384->ctx.mode == ESP32_SHA_HW) {
         /* release hw */
-        esp_sha_hw_unlock(&(sha512->ctx));
+        esp_sha_hw_unlock(&(sha384->ctx));
     }
     /* always set mode as INIT
     *  whether using HW or SW is determined at first call of update()

wolfSSH has been updated to tag v1.4.6-stable
mongoose has not been updated but needs the following patch:

diff --git a/mongoose.c b/mongoose.c
index b12cff18..60a7f62e 100644
--- a/mongoose.c
+++ b/mongoose.c
@@ -9160,7 +9160,7 @@ static void mg_send_file_data(struct mg_connection *nc, FILE *fp) {
 static void mg_do_ssi_include(struct mg_connection *nc, struct http_message *hm,
                               const char *ssi, char *tag, int include_level,
                               const struct mg_serve_http_opts *opts) {
-  char file_name[MG_MAX_PATH], path[MG_MAX_PATH], *p;
+  char file_name[MG_MAX_PATH], path[MG_MAX_PATH+2], *p;
   FILE *fp;
 
   /*
diff --git a/mongoose.h b/mongoose.h
index 3bcf8147..5649e1a7 100644
--- a/mongoose.h
+++ b/mongoose.h
@@ -1768,7 +1768,7 @@ typedef struct {
 
 void cs_md5_init(cs_md5_ctx *c);
 void cs_md5_update(cs_md5_ctx *c, const unsigned char *data, size_t len);
-void cs_md5_final(unsigned char *md, cs_md5_ctx *c);
+void cs_md5_final(unsigned char md[16], cs_md5_ctx *c);
 
 #ifdef __cplusplus
 }

To be able to "test" this PR, you'll need:

An ESP-IDF v5 installation (from git) tested and configured, set to tag v5.0

Ideally this may need to be broken down for better review - note that you should be able to review commit per commit as I tried to keep them consistent and compilable.

llange · 2023-01-31T22:19:58Z

Cross-referencing #360 and #263

jetpax · 2023-03-04T04:59:24Z

Thanks, this is a great step forward, and works pretty well for me with esp-idf=5.0.1, given your caveats, as long as I have CONFIG_OVMS_COMP_CELLULAR=y in sdkconfig.

Without that, it fails to compile with

.../main/ovms_command.cpp:1249:15: error: aggregate 'stat st' has incomplete type and cannot be defined
 1249 |   struct stat st;

Not clear to me where this error is coming from, any ideas?

llange · 2023-03-05T20:08:12Z

Hello @jetpax,
Thank you very much for taking the time (and the risk...) to test this branch. I'm glad that you managed to have it work, and I look forward to your opening of many issue tickets so that we can improve it.
I confirm that CONFIG_OVMS_COMP_CELLULAR is y in my builds, and I'll check why it doesn't compile without it. Stay tuned !

llange · 2023-03-05T20:37:28Z

I added a missing header, which fixes the compilation. Tell me if it works for you !
(This branch is updated with the change, you can pull it to retrieve it)

jetpax · 2023-03-06T17:33:31Z

Thanks @llange , that works great, and system seems much more stable.

I do also get a compile error when I tried to turn off Duktape with CONFIG_OVMS_SC_JAVASCRIPT_NONE=y

In file included from /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/powermgmt/powermgmt_web.cpp:31:
/Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_metrics.h:168:29: error: 'OvmsWriter' was not declared in this scope
  168 | int OvmsMetricUnit_Validate(OvmsWriter* writer, int argc, const char* token, bool complete, metric_group_t group = GrpNone);

The reason I was doing that was that I had a crash that I though might have been memory allocation related when I tried to turn on CAN tcpserver network streaming to use SavvyCAN, (though maybe Mongoose related?)

In case it's of interest, here's the commands to replicate that crash:

OVMS> enable
Password:
Secure mode
OVMS# can can1 start listen 500000
Can bus can1 started in mode listen at speed 500000bps
I (193297) events: Signal(power.can1.on)
OVMS# can log start tcpserver discard gvret-b :23
CAN logging as TCP server: Type:tcpserver Format:gvret-b(discard) Vehicle:DEMO Path::23
I (205507) canlog-tcpserver: Launching TCP server at :23
I (273977) canlog-tcpserver: Log service connection from 192.168.1.153

Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x400fe49a  PS      : 0x00060630  A0      : 0x800fe514  A1      : 0x3fff89c0  
A2      : 0x3ffb5da0  A3      : 0x3fff89e0  A4      : 0x3ffff124  A5      : 0x3fff4430  
A6      : 0x00000003  A7      : 0x00060023  A8      : 0x00000000  A9      : 0x00000000  
A10     : 0x00000000  A11     : 0x00000001  A12     : 0x3ffb5da8  A13     : 0x00000001  
A14     : 0x3ff5f07c  A15     : 0x00000001  SAR     : 0x00000013  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000010  LBEG    : 0x4008b86d  LEND    : 0x4008b87d  LCOUNT  : 0xfffffffc  


Backtrace: 0x400fe497:0x3fff89c0 0x400fe511:0x3fff89e0 0x401026ef:0x3fff8a20 0x4010283b:0x3fff8aa0 0x4013beb9:0x3fff8ac0 0x4013d21a:0x3fff8b00 0x4013d5da:0x3fff8b20 0x4013d8a2:0x3fff8b80 0x4013a70d:0x3fff8be0 0x400edb47:0x3fff8c00 0x400edbe5:0x3fff8c50

Here's the Stack Trace decode

0x400fe49a: canformat_gvret_binary::PopulateBusList12(gvret_replymsg_t*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canformat_gvret.cpp:401
0x400fe497: canformat_gvret_binary::PopulateBusList12(gvret_replymsg_t*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canformat_gvret.cpp:401
0x400fe511: canformat_gvret_binary::getheader[abi:cxx11](timeval*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canformat_gvret.cpp:223
0x401026ef: canlog_tcpserver::MongooseHandler(mg_connection*, int, void*) at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canlog_tcpserver.cpp:236
0x4010283b: tsMongooseHandler at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/can/src/canlog_tcpserver.cpp:129
0x4013beb9: mg_call at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:2368
0x4013d21a: mg_if_accept_tcp_cb at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:2764
0x4013d5da: mg_accept_conn at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:3578
0x4013d5da: mg_mgr_handle_conn at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:3842
0x4013d8a2: mg_socket_if_poll at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:4042
0x4013a70d: mg_mgr_poll at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/components/mongoose/mongoose/mongoose.c:2538
0x400edb47: OvmsNetManager::MongooseTask() at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:772
0x400edbe5: MongooseRawTask at /Users/jep/github/Open-Vehicle-Monitoring-System-3/vehicle/OVMS.V3/main/ovms_netmanager.cpp:754

llange · 2023-03-10T12:28:31Z

Hi @jetpax , thanks for the bug report !
I'll try to reproduce it to see how we can mitigate the crash and hopefully fix it.

Regarding the compilation error with CONFIG_OVMS_SC_JAVASCRIPT_NONE=y ; I confirm it happens also on my side, and it's not that easy to fix at the moment. I'll keep you informed when I have a working solution.
Did you try this CONFIG_OVMS_SC_JAVASCRIPT_NONE=y with the (current) master branch ? Does it compile OK ?

Signed-off-by: Ludovic LANGE <[email protected]>

wolfssl has been forked (https://github.com/openvehicles/wolfssl) and the subdirectory converted to a Git submodule. Additionally, it has been put one directory level below (in vehicle/OVMS.V3/components/wolfssl/wolfssl) to better separate the upstream code from our build glue (component.mk, CMakeLists.txt and configuration files) This way it's easier to track any changes made to the module, upgrade when upstram changes, or propose patches to upstream. We kept the existing version (v4.7.0-stable) - it will be upgraded later Signed-off-by: Ludovic LANGE <[email protected]>

Signed-off-by: Ludovic LANGE <[email protected]>

There are some issues with newer versions (crash of the SSH console) so we keep v5.3.0 at the moment Signed-off-by: Ludovic LANGE <[email protected]>

Signed-off-by: Ludovic LANGE <[email protected]>

llange force-pushed the experimental-esp-idf-v5 branch from 4961f93 to ca0b2dc Compare January 31, 2023 00:46

llange changed the title ~~Experimental ESP-IDF v4~~ Experimental ESP-IDF v5 Jan 31, 2023

llange force-pushed the experimental-esp-idf-v5 branch from ca0b2dc to 2e25d8a Compare January 31, 2023 22:05

This was referenced Feb 2, 2023

replace some constants with their more adapted counterpart #813

Merged

Add header and fix function signature #815

Merged

llange force-pushed the experimental-esp-idf-v5 branch 2 times, most recently from 1402b05 to 1311f4e Compare February 6, 2023 21:14

llange force-pushed the experimental-esp-idf-v5 branch 5 times, most recently from 1ad89a2 to ef58ea9 Compare February 24, 2023 15:42

llange force-pushed the experimental-esp-idf-v5 branch 4 times, most recently from e54513f to 9f88281 Compare February 27, 2023 23:15

llange mentioned this pull request Mar 5, 2023

add a missing include #850

Merged

llange force-pushed the experimental-esp-idf-v5 branch from aeb7813 to c28b92f Compare March 11, 2023 19:42

llange force-pushed the experimental-esp-idf-v5 branch 2 times, most recently from 951114b to 3919e8e Compare March 27, 2023 20:41

llange force-pushed the experimental-esp-idf-v5 branch 4 times, most recently from b092373 to 01f95d3 Compare April 10, 2023 20:52

llange force-pushed the experimental-esp-idf-v5 branch 6 times, most recently from 3197172 to 42e0da6 Compare April 23, 2023 19:03

llange force-pushed the experimental-esp-idf-v5 branch 3 times, most recently from f13707a to e202470 Compare May 2, 2023 14:23

llange force-pushed the experimental-esp-idf-v5 branch from e202470 to fa45d7c Compare January 16, 2024 23:21

WIP WIP WIP : prevent a crash at boot (to be analysed)

112b33d

Signed-off-by: Ludovic LANGE <[email protected]>

llange force-pushed the experimental-esp-idf-v5 branch from fa45d7c to d217ae2 Compare April 5, 2024 22:14

llange added 5 commits April 6, 2024 00:47

ESP-IDF v5+: wolfssl : use globbing and can use latest version

ba60041

Signed-off-by: Ludovic LANGE <[email protected]>

ESP-IDF v5+: wolfssl : update user_settings.h (disable OPENSSL_*)

645f673

Signed-off-by: Ludovic LANGE <[email protected]>

update wolfSSL to v5.3.0-stable

6ba36f4

There are some issues with newer versions (crash of the SSH console) so we keep v5.3.0 at the moment Signed-off-by: Ludovic LANGE <[email protected]>

WIP WIP WIP : Adding notice in README.md

dc0b064

Signed-off-by: Ludovic LANGE <[email protected]>

llange force-pushed the experimental-esp-idf-v5 branch from d217ae2 to dc0b064 Compare April 5, 2024 22:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental ESP-IDF v5 #810

Experimental ESP-IDF v5 #810

llange commented Jan 31, 2023 •

edited

Loading

llange commented Jan 31, 2023

jetpax commented Mar 4, 2023

llange commented Mar 5, 2023

llange commented Mar 5, 2023

jetpax commented Mar 6, 2023 •

edited

Loading

llange commented Mar 10, 2023

Experimental ESP-IDF v5 #810

Are you sure you want to change the base?

Experimental ESP-IDF v5 #810

Conversation

llange commented Jan 31, 2023 • edited Loading

llange commented Jan 31, 2023

jetpax commented Mar 4, 2023

llange commented Mar 5, 2023

llange commented Mar 5, 2023

jetpax commented Mar 6, 2023 • edited Loading

llange commented Mar 10, 2023

llange commented Jan 31, 2023 •

edited

Loading

jetpax commented Mar 6, 2023 •

edited

Loading