Use of custom dialect makes node.Close() hang

[JonPichel]

Hi,

I am using 3DR SiK radios. I am noticing weird behavior when using custom dialects. After the program finishes, when I try to close the nodes, the call to node.Close hangs.

I provide an example of what I am talking about. When using the custom dialect (taken from the library examples), after sending 5 messages the program hangs on the close calls. When using the ardupilotmega one, it doesn't.

package main

import (
	"fmt"
	"log"
	"sync"
	"time"

	"github.com/bluenviron/gomavlib/v3"
	"github.com/bluenviron/gomavlib/v3/pkg/dialect"
	"github.com/bluenviron/gomavlib/v3/pkg/dialects/ardupilotmega"
	"github.com/bluenviron/gomavlib/v3/pkg/message"
)

type MessageCustom struct {
	Param1 uint8
	Param2 uint8
	Param3 uint32
}

func (*MessageCustom) GetID() uint32 {
	return 22
}

var (
	myDialect = &dialect.Dialect{
		Version: 3,
		Messages: []message.Message{
			&MessageCustom{},
		},
	}
)

func main() {
	// Select between custom dialect or ardupilotmega
	custom := true

	var dialect *dialect.Dialect
	if custom {
		dialect = myDialect
	} else {
		dialect = ardupilotmega.Dialect
	}

	senderNode, err := gomavlib.NewNode(gomavlib.NodeConf{
		Endpoints: []gomavlib.EndpointConf{
			gomavlib.EndpointSerial{
				Device: "/dev/ttyUSB4",
				Baud:   57600,
			},
		},
		Dialect:     dialect,
		OutVersion:  gomavlib.V2,
		OutSystemID: 10,
	})
	if err != nil {
		log.Fatal("Creating receiver node")
	}
	defer senderNode.Close()

	receiverNode, err := gomavlib.NewNode(gomavlib.NodeConf{
		Endpoints: []gomavlib.EndpointConf{
			gomavlib.EndpointSerial{
				Device: "/dev/ttyUSB5",
				Baud:   57600,
			},
		},
		Dialect:     dialect,
		OutVersion:  gomavlib.V2,
		OutSystemID: 11,
	})
	if err != nil {
		log.Fatal("Creating receiver node")
	}
	defer receiverNode.Close()

	done := make(chan struct{})
	var wg sync.WaitGroup
	wg.Add(2)
	go func() {
	loop:
		for {
			select {
			case evt := <-senderNode.Events():
				if frm, ok := evt.(*gomavlib.EventFrame); ok {
					fmt.Printf("[TX-RX] %T\n", frm.Message())
				}
			case <-done:
				break loop
			}
		}
		wg.Done()
	}()
	go func() {
	loop:
		for {
			select {
			case evt := <-receiverNode.Events():
				if frm, ok := evt.(*gomavlib.EventFrame); ok {
					fmt.Printf("[RX] id=%d, %+v\n", frm.Message().GetID(), frm.Message())
				}
			case <-done:
				break loop
			}
		}
		wg.Done()
	}()

	for range 5 {
		if custom {
			senderNode.WriteMessageAll(&MessageCustom{
				Param1: 1,
				Param2: 2,
				Param3: 3,
			})
		} else {
			senderNode.WriteMessageAll(&ardupilotmega.MessageParamValue{
				ParamId:    "test_parameter",
				ParamValue: 123456,
				ParamType:  ardupilotmega.MAV_PARAM_TYPE_UINT32,
				ParamCount: 1,
				ParamIndex: 1,
			})
		}
		time.Sleep(time.Second)
	}

	fmt.Println("CLOSING channel")
	close(done)
	wg.Wait()
	fmt.Println("GOODBYE")
}

EDIT: When I first opened the issue I thought this was caused by opening two nodes from a single process, but after further testing I noticed the issue only happened when using a custom dialect.

[aler9]

Hello, if a node is not closing, it probably means that the serial channel is not closing, i.e. this line is not returning:

gomavlib/endpoint_serial.go

Line 60 in b0877b5

t.reconnector.Close()

This might be influenced by the quantity of bytes you're feeding into the serial port, and consequently, by messages and dialects, but just in terms of size, not content. Dialects have no effect on routines and are just converters that allow to convert Golang structs to bytes and vice-versa.

In order to further debug this issue, you need to provide a list of all open goroutines during the freeze, that can be generated in this way:

1. add this somewhere

// inside the import section
import (
_ "net/http/pprof"
)

// inside main
go func(mux *http.ServeMux) {
    (&http.Server{
        Addr: ":9999",
        Handler: mux,
    }).ListenAndServe()
}(http.DefaultServeMux)
http.DefaultServeMux = http.NewServeMux()

2. start the program and wait for the freeze to happen

3. dump the goroutine stack

go tool pprof -text http://localhost:9999/debug/pprof/goroutine

4. attach here

[aler9]

closed since no feedback was provided.