Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use go version 1.20 to build and test FDW #296

Open
pskrbasu opened this issue Mar 3, 2023 · 3 comments
Open

Use go version 1.20 to build and test FDW #296

pskrbasu opened this issue Mar 3, 2023 · 3 comments
Assignees
Labels
github_actions Pull requests that update GitHub Actions code

Comments

@pskrbasu
Copy link
Contributor

pskrbasu commented Mar 3, 2023

No description provided.

@pskrbasu pskrbasu added the github_actions Pull requests that update GitHub Actions code label Mar 3, 2023
@pskrbasu pskrbasu self-assigned this Mar 3, 2023
@pdecat
Copy link
Contributor

pdecat commented Mar 21, 2023

FWIW, I had issues when building steampipe-postgres-fdw using the current golang:1.20 docker image (shared tag currently pointing to 1.20.2-bullseye for Linux).

Error at runtime is undefined symbol: __res_search:

# steampipe --install-dir /var/task/.steampipe plugin list
...
2023-03-21 14:23:58.671 UTC [TRACE] steampipe: Pinging
Error: plugin listing failed - could not load library "/var/task/.steampipe/db/14.2.0/postgres/lib/postgresql/steampipe_postgres_fdw.so": /var/task/.steampipe/db/14
.2.0/postgres/lib/postgresql/steampipe_postgres_fdw.so: undefined symbol: __res_search (SQLSTATE 58P01)

That symbol is provided by libresolv:

root@d9b971426328:/app# grep __res_search /usr/lib/x86_64-linux-gnu/*.so
grep: /usr/lib/x86_64-linux-gnu/libresolv.so: binary file matches

Inspecting the binary compiled with go 1.20:

root@d9b971426328:/app# nm build-Linux/steampipe_postgres_fdw.so |grep __res_search
                 U __res_search

root@d9b971426328:/app# ldd build-Linux/steampipe_postgres_fdw.so
        linux-vdso.so.1 (0x00007ffe477d2000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffb79423000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ffb7b736000)

Comparing that to the binary compiled with go 1.19, we can see that this symbol is not used:

root@c70a8b6ba81f:/app# nm build-Linux/steampipe_postgres_fdw.so | grep __res_search

root@c70a8b6ba81f:/app# ldd build-Linux/steampipe_postgres_fdw.so
        linux-vdso.so.1 (0x00007ffdeb88b000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fceef8ef000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fcef1bf4000)

Downgrading to golang:1.19 docker image resolves the issue.

Something changed on that front when using -buildmode=c-archive with go 1.20, but supposedly only for macOS according to golang/go#58159

@pdecat
Copy link
Contributor

pdecat commented Mar 21, 2023

Defining SHLIB_LINK="-lresolv" seem to do the trick.

Edit: it does not work because all symbols from steampipe_postgres_fdw.a are dropped.

Without it:

root@387a98fca485:/app# go version
go version go1.20.2 linux/amd64

root@387a98fca485:/app# ./build_binary.sh
rm -f steampipe_postgres_fdw.so   libsteampipe_postgres_fdw.a  libsteampipe_postgres_fdw.pc
rm -f datum.o query.o fdw.o logging.o datum.bc query.bc fdw.bc logging.bc
rm -rf steampipe_postgres_fdw.a fdw.h
rm -rf results/ regression.diffs regression.out tmp_check/ tmp_check_iso/ log/ output_iso/
go build -o steampipe_postgres_fdw.a -buildmode=c-archive ../*.go
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o datum.o datum.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o query.o query.c
In file included from /usr/local/include/postgresql/server/postgres.h:47,
                 from fdw_helpers.h:2,
                 from fdw.go:6,
                 from query.c:1:
query.c: In function 'canonicalOpExpr':
query.c:227:77: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
  227 |     elog(DEBUG5, "canonicalOpExpr, arg length: %d, base_relids %x", length, (int)base_relids);
      |                                                                             ^
/usr/local/include/postgresql/server/utils/elog.h:141:4: note: in definition of macro 'ereport_domain'
  141 |    __VA_ARGS__, errfinish(__FILE__, __LINE__, PG_FUNCNAME_MACRO); \
      |    ^~~~~~~~~~~
/usr/local/include/postgresql/server/utils/elog.h:233:2: note: in expansion of macro 'ereport'
  233 |  ereport(elevel, errmsg_internal(__VA_ARGS__))
      |  ^~~~~~~
query.c:227:5: note: in expansion of macro 'elog'
  227 |     elog(DEBUG5, "canonicalOpExpr, arg length: %d, base_relids %x", length, (int)base_relids);
      |     ^~~~
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o fdw.o fdw.c
fdw.c: In function 'fdwGetForeignRelSize':
fdw.c:105:5: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
  105 |     FdwPlanState *planstate = palloc0(sizeof(FdwPlanState));
      |     ^~~~~~~~~~~~
fdw.c: In function 'fdwGetForeignPlan':
fdw.c:291:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
  291 |  ForeignScan * s = make_foreignscan(
      |  ^~~~~~~~~~~
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o logging.o logging.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -shared -o steampipe_postgres_fdw.so datum.o query.o fdw.o logging.o -L/usr/local/lib    -Wl,--as-needed -Wl,-rpath,'/usr/local/lib',--enable-new-dtags  steampipe_postgres_fdw.a
mkdir -p ../build-Linux
rm -f ../build-Linux/*
cp steampipe_postgres_fdw.so ../build-Linux
cp steampipe_postgres_fdw.control ../build-Linux
cp steampipe_postgres_fdw--1.0.sql ../build-Linux
rm steampipe_postgres_fdw.so
rm steampipe_postgres_fdw.a
rm steampipe_postgres_fdw.h
rm ./*.o
/app

root@387a98fca485:/app# nm build-Linux/steampipe_postgres_fdw.so | grep __res_search
                 U __res_search

With it:

root@387a98fca485:/app# go version
go version go1.20.2 linux/amd64

root@387a98fca485:/app# cd fdw

root@387a98fca485:/app/fdw# make clean
rm -f steampipe_postgres_fdw.so   libsteampipe_postgres_fdw.a  libsteampipe_postgres_fdw.pc
rm -f datum.o query.o fdw.o logging.o datum.bc query.bc fdw.bc logging.bc
rm -rf steampipe_postgres_fdw.a fdw.h
rm -rf results/ regression.diffs regression.out tmp_check/ tmp_check_iso/ log/ output_iso/

root@387a98fca485:/app/fdw# make go
go build -o steampipe_postgres_fdw.a -buildmode=c-archive ../*.go

root@387a98fca485:/app/fdw# make SHLIB_LINK="-lresolv"
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o datum.o datum.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o query.o query.c
In file included from /usr/local/include/postgresql/server/postgres.h:47,
                 from fdw_helpers.h:2,
                 from fdw.go:6,
                 from query.c:1:
query.c: In function 'canonicalOpExpr':
query.c:227:77: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
  227 |     elog(DEBUG5, "canonicalOpExpr, arg length: %d, base_relids %x", length, (int)base_relids);
      |                                                                             ^
/usr/local/include/postgresql/server/utils/elog.h:141:4: note: in definition of macro 'ereport_domain'
  141 |    __VA_ARGS__, errfinish(__FILE__, __LINE__, PG_FUNCNAME_MACRO); \
      |    ^~~~~~~~~~~
/usr/local/include/postgresql/server/utils/elog.h:233:2: note: in expansion of macro 'ereport'
  233 |  ereport(elevel, errmsg_internal(__VA_ARGS__))
      |  ^~~~~~~
query.c:227:5: note: in expansion of macro 'elog'
  227 |     elog(DEBUG5, "canonicalOpExpr, arg length: %d, base_relids %x", length, (int)base_relids);
      |     ^~~~
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o fdw.o fdw.c
fdw.c: In function 'fdwGetForeignRelSize':
fdw.c:105:5: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
  105 |     FdwPlanState *planstate = palloc0(sizeof(FdwPlanState));
      |     ^~~~~~~~~~~~
fdw.c: In function 'fdwGetForeignPlan':
fdw.c:291:2: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
  291 |  ForeignScan * s = make_foreignscan(
      |  ^~~~~~~~~~~
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -I. -I./ -I/usr/local/include/postgresql/server -I/usr/local/include/postgresql/internal  -D_GNU_SOURCE   -c -o logging.o logging.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wno-format-truncation -Wno-stringop-truncation -O2 -I/usr/local/include/server -I/usr/local/include/internal -g -fPIC -shared -o steampipe_postgres_fdw.so datum.o query.o fdw.o logging.o -L/usr/local/lib    -Wl,--as-needed -Wl,-rpath,'/usr/local/lib',--enable-new-dtags  -lresolv

root@387a98fca485:/app/fdw# nm steampipe_postgres_fdw.so | grep __res_search

root@387a98fca485:/app/fdw# make inst
mkdir -p ../build-Linux
rm -f ../build-Linux/*
cp steampipe_postgres_fdw.so ../build-Linux
cp steampipe_postgres_fdw.control ../build-Linux
cp steampipe_postgres_fdw--1.0.sql ../build-Linux
rm steampipe_postgres_fdw.so
rm steampipe_postgres_fdw.a
rm steampipe_postgres_fdw.h
rm ./*.o
root@387a98fca485:/app/fdw# cd ..

root@387a98fca485:/app# nm build-Linux/steampipe_postgres_fdw.so | grep __res_search

@binaek
Copy link
Contributor

binaek commented Aug 17, 2023

Handled by #339

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github_actions Pull requests that update GitHub Actions code
Projects
None yet
Development

No branches or pull requests

3 participants