Calling otCommissionerStop as part of OT_COMMISSIONER_JOINER_REMOVED process result is failed joining

[rretanubun]

My setup is an 2x FTD on git commit 8f92d2d

I am trying to automate commissioner stop part of the CLI after all the joiners in commissioner joiner table is joined.

When HandleJoinerEvent gets a OT_COMMISSIONER_JOINER_REMOVED event and the code sense (using local counters) that there is no more joiner entries it calls otCommissionerStop.

However, doing this results in joiners being unable to join

Tracer from the Joiner Side

> eui64
04cd15fffe7ab28e
Done
> JOINER: Started
[N] MeshForwarder-: Failed to send IPv6 UDP msg, len:56, chksum:d33d, ecn:no, to:None, sec:no, error:Abort, prio:net
[N] MeshForwarder-:     src:[fe80:0:0:0:3c76:204f:2b19:2b7c]:19788
[N] MeshForwarder-:     dst:[ff02:0:0:0:0:0:0:2]:19788
[-] --------------: =========[[THCI] direction=send | type=JOIN_FIN.req | len=017]==========
[-] --------------: | 10 01 01 21 00 22 00 23 | 00 25 06 18 B4 30 00 00 | ...!.".#.%..40..
[-] --------------: | 10 .. .. .. .. .. .. .. | .. .. .. .. .. .. .. .. | ................
[-] --------------: ------------------------------------------------------------------------
[N] CoapSecure----: Transmit: InvalidArgs
JOINER: CB ERR: 28, ResponseTimeout
[W] Mle-----------: Failed to process UDP: InvalidState

Tracer from the Commissioner Side

esp32> 
esp32> ot-ctl commissioner state
coesp32> mmissioner state
active
Done
> ot-ctl commissioner joiner table
cesp32> ommissioner joiner table
| ID                    | PSKd                             | Expiration |
+-----------------------+----------------------------------+------------+
|      04cd15fffe7ab28e |                           J01NME |    3583661 |
Done
> ~ Discovery Request from 3e76204f2b192b7c: version=4,joiner=1
I (15:33:31.114) allow_list: Start Joiner
I (15:33:31.114) OPENTHREAD: Platform UDP bound to port 49155
I (15:33:36.058) allow_list: Connect Joiner
I (15:33:36.089) allow_list: Finalize Joiner
I (15:33:36.090) allow_list: Removed Joiner
I (15:33:36.091) allow_list: Calling otCommissionerStop
I (15:33:36.094) OPENTHREAD: Platform UDP bound to port 49156
I(139498) OPENTHREAD:[N] MeshForwarder-: Dropping rx lowpan HC frame, error:Drop, len:40, src:4306cd6fc57dd53e, dst:6e036f58d14726a0, sec:no
I(139508) OPENTHREAD:[N] MeshForwarder-: Dropping rx lowpan HC frame, error:Drop, len:40, src:4306cd6fc57dd53e, dst:6e036f58d14726a0, sec:no

If I remove the otCommissionerStop call and just manually stop the commissioner (at human speed), everything works fine.

I also notice that the time gap between Finalize and Remove is very fast ~1ms?
I was expecting it to follow kRemoveJoinerDelay (set to 20)
https://github.com/openthread/openthread/blob/main/src/core/meshcop/commissioner.cpp#L1115

Is there a different mechanism / callback point that can be used to programmatically
stop the commissioner at the proper time that I am missing?

[rretanubun]

Well, this was self inflicted it turns out. For anyone else finding themselves here, In my case, my root cause was that I am calling otCommissionerRemoveJoiner during the OT_COMMISSIONER_JOINER_FINALIZE callback, which is not the proper thing to do.