CopyFrom appears to slow down in v5

[nassibnassar]

Thank you for this amazingly useful toolkit. I am sure the problem below is user error on my part. I have code that calls CopyFrom with ~600 million rows read from many gob-encoded files on local disk. If I change the pgx version from v4 (e.g. v4.17.2) to v5.1.1, the running time increases from 0.4 hours to 1.8 hours. This is the Values() method:

func (s *Source) Values() ([]any, error) {
        var err error
        switch {
        case s.err != nil:
                return nil, s.err
        case s.record == nil:
                s.err = fmt.Errorf("no record available: %s", s.path)
                return nil, s.err
        default:
                var r = s.record
                var srsID, matchedID, instanceID pgtype.UUID
                if srsID, err = uuid.EncodeUUID(r.SRSID); err != nil {
                        return nil, fmt.Errorf("encoding srs_id: %v", err)
                }
                if matchedID, err = uuid.EncodeUUID(r.MatchedID); err != nil {
                        return nil, fmt.Errorf("encoding matched_id: %v", err)
                }
                if instanceID, err = uuid.EncodeUUID(r.InstanceID); err != nil {
                        s.printerr("id=%s: encoding instance_id %q: %v", r.SRSID, r.InstanceID, err)
                        instanceID = uuid.EncodeNilUUID()
                }
                var v = []any{
                        srsID,
                        r.Line,
                        matchedID,
                        r.InstanceHRID,
                        instanceID,
                        r.Field,
                        r.Ind1,
                        r.Ind2,
                        r.Ord,
                        r.SF,
                        r.Content,
                }
                return v, nil
        }
}

Source: https://github.com/library-data-platform/ldpmarc/blob/db78cb9dc6069604cc2be1c8ef3a414128212da3/cmd/ldpmarc/local/local.go
Called from: https://github.com/library-data-platform/ldpmarc/blob/db78cb9dc6069604cc2be1c8ef3a414128212da3/cmd/ldpmarc/main.go

The data types used are int16, string, and pgtype.UUID. This is running with Go 1.19. Thank you for any help you can offer.

[jackc]

As far as I know there shouldn't be any significant performance changes in the copy system itself. If I had to guess I would say it is something with the UUIDs.

Here's some things to try:

1. See if you can run the copy without the UUID columns and see if there are any observable differences in performance.
2. pprof would probably find the culprit -- whatever it is.
3. Make sure when you move to v5 you are also moving to the pgtype now merged into v5. The old pgtype might still work but it probably would be jumping through extra hoops.
4. If it is the UUID, you might be want to use google/uuid or gofrs/uuid instead of strings to the UUIDs. They may be faster at parsing the string UUIDs.

[nassibnassar]

Using the pgtype merged into v5 solved the problem. Many thanks!