OTBR unable to ping on-link external network

[RahulRGaikwad]

Hello everyone,

Facing a problem in getting to my OTBR to ping external network devices(Wi-Fi side).

Have built the default example 'ot_br', using ESP-IDF tag:v5.1.1 on ESP32-C6(native radio).

Have enabled AutoStart and set my Wi-Fi credentials using idf.py menuconfig.

To verify, I tried to ping back and forth between my OTBR, and Linux Host.
Host->OTBR works fine, but fails vice-versa.

Using Wireshark, I monitored traffic on host side, and looks like in both cases, there was a Echo Request/Reply pair.

So I enabled Ip6 Logs on LWIP layer, and it looks the Echo request got forwarded to 'sta' netif, and also recieved a Echo Reply on 'ot' netif.
The ip6_debug message looks weird though. The logs are attached below

I(180278) OPENTHREAD:[I] Icmp6---------: Sent echo request: (seq = 9)
ip6_input: packet accepted on interface XX
ip6_input: packet not for us.
ip6_forward: forwarding packet to fd37:7b4e:3a09:c00a:12a:7d30:fa05:ca6a
ip6_input: packet accepted on interface ot
ip6_input:
IPv6 header:
+-------------------------------+
| 8 | 16 | 51198 | (ver, class, flow)
+-------------------------------+
| 9 | 0 | 9 | (plen, nexth, hopl)
+-------------------------------+
| 2 | c03c | 0 | 0 | (src)
| 0 | 100 | 0 | 0 |
+-------------------------------+
| 0 | 0 | 0 | 0 | (dest)
| 0 | 0 | 3d | 160b |
+-------------------------------+
ip6_input: p->len 16 p->tot_len 16

Any idea on what could be wrong?

Thanks in advance,
Rahul

[zwx1995esp]

Hi, @RahulRGaikwad I don't think this is an issue, and here is the explaination:

Our ESP BR framework is like this, OT stack and Wi-Fi stack are trated as one netif(net interface) glued to lwip, and the address used by OT and Wi-Fi will also assigned on LWIP:

 -------                                                  ------------
 |     |                                                  |  LWIP    |
 |  H  |                                                  |   -----  |
 |  O  |                                                  |   |   |  |
 |  S  |                                                  |-↑-↓---↑--|
 |  T  | <---BR sends the Ping Request to Host --------   | |   | |  |
 |     | ----Host sends the Ping Reply back to BR-------->|-|   | |--|<------Ping command is 
 |     |                                                  | WiFi| OT |       implemented at the OT layer
 |     |                                                  |     |    |
 -------                                                  ------------

LWIP is the upper layer of the OT and Wi-Fi, which is used to forward the packets from each netifs. The command ping you used is implemented in OT stack layer, and in our BR, we need to send the ping packets to LWIP layer, the LWIP layer will forwarding this packet to Wi-Fi.

In your scenario: the ping command is started at the OT layer, and then send to LWIP to forward. LWIP finds the dest address is on the Wi-Fi netif, so it forward this packets to Wi-Fi, and sends it on the Wi-Fi interface. The host received this packets and then send the reply to BR. This packet will be received on the Wi-Fi inteface by BR, and push the packet to LWIP to forward. LWIP find the dest address is us(OT layer will assign the OMR address on the LWIP), so LWIP will not forward it back to OT. And OT layer ping shows that the reply is loss, but the reply packet does receive by BR.

To solve it(I mean that just see the ping successful result from BR to Host), you can implement a LWIP layer ping command, and using this command to ping Host.(I have already tried this experiment before, using the LWIP socket to implement a Ping comand, and it did ping from BR to host).

[RahulRGaikwad]

Our ESP BR framework is like this, OT stack and Wi-Fi stack are trated as one netif(net interface) glued to lwip, and the address used by OT and Wi-Fi will also assigned on LWIP:

I assume you mean to say that both OpenThread and Wi-Fi have their own netifs.

I understand do what you are saying, as I have gone through the code as well.Then, I have to treat the OT stack to do only lower layer activities, right?

This affects the other use cases of OT stack, like UDP or TCP right?

Can we not use the IP6_INPUT_HOOK, to relay the message back to OT stack using Error Ip6::HandleDatagram() ?

[chshu]

@RahulRGaikwad

Then, I have to treat the OT stack to do only lower layer activities, right?
This affect the other use cases of OT stack, like UDP or TCP right?

Yes, OT is considered as a lower layer interface, the same to Wi-Fi. Applications should use LwIP UDP/TCP/ICMP APIs.

Can we not use the IP6_INPUT_HOOK, to relay the message back to OT stack using Error Ip6::HandleDatagram() ?

LwIP assumes that the message has already reached its final destination, so it doesn't forward the message to OT stack, no matter it's netif_input or HandleDatagram, right?

It is a known phenomenon when pinging the host from the OT CLI in the ESP Thread BR. However, this problem does not happen when using LwIP ping or when pinging between the host and Thread devices through BR.

[chshu]

@RahulRGaikwad BTW, since only one RF path in ESP32-C6, which means Wi-Fi and Thread can't receive simultaneously, it has a significant impact on performance. Hence the two SoCs solution is recommended for Thread BR production. You may refer to the follow links for more details:

• https://github.com/espressif/esp-thread-br
• https://docs.espressif.com/projects/esp-thread-br/en/latest/hardware_platforms.html#hardware-platforms

[zwx1995esp]

Can we not use the IP6_INPUT_HOOK, to relay the message back to OT stack using Error Ip6::HandleDatagram() ?

As chshu said, we do not recommend to use the OT ping command to ping from BR to Host, because OT ping will choose the OT OMR address as the source address. But for your case ping from BR to Host, using a Wi-Fi netif address as the source address on BR side is more reasonable. LWIP ping can auto choose that.

This scenario is more like running OT-BR-posix on linux, we prefer to use the linux ping command to ping Host rather than use a OT ping command(like ot-cli ping).

[RahulRGaikwad]

@chshu

Yes, OT is considered as a lower layer interface, the same to Wi-Fi. Applications should use LwIP UDP/TCP/ICMP APIs.

Alright, thanks for the confirmation. Will try out communicating with LWIP.

LwIP assumes that the message has already reached its final destination, so it doesn't forward the message to OT stack, no matter it's netif_input or HandleDatagram, right?

The Reply has indeed reached it's destination, but as the upper layer handlers are present in OT stack rather than the LWIP Stack, the reply is kind of lost.
That's why I suggested if we could find a way to handle which stack the message needs to go into, using the Error Ip6::HandleDatagram() in the IP6_INPUT_HOOK. We only pass on data-grams, that do not belong to the LWIP stack, or something like that.
This way I thought the OT Stack Upper Level API could work fine.

It is a known phenomenon when pinging the host from the OT CLI in the ESP Thread BR. However, this problem does not happen when using LwIP ping or when pinging between the host and Thread devices through BR.

BTW, is this mentioned somewhere in docs. If so, I would appreciate it if you linked it here.

[RahulRGaikwad]

@zwx1995esp

As chshu said, we do not recommend to use the OT ping command to ping from BR to Host, because OT ping will choose the OT OMR address as the source address. But for your case ping from BR to Host, using a Wi-Fi netif address as the source address on BR side is more reasonable. LWIP ping can auto choose that.

That makes sense. I was just messing around with OT CLI variations. I will use this approach normally though, thanks.

[RahulRGaikwad]

Was able to send messages, using Openthread stack as a lower layer stack, and LWIP for the upper layers.