Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-979081 Use libsnowflakeclient as node js driver base #853

Draft
wants to merge 50 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
35032ad
SNOW-979081: Init libsnowflakeclient on darwin with gyp
sfc-gh-dprzybysz May 29, 2024
1f16342
SNOW-979081: Login and execute quest
sfc-gh-dprzybysz May 29, 2024
a61c23a
SNOW-979081: Separate login from execute query
sfc-gh-dprzybysz May 29, 2024
7b7a769
SNOW-979081: Execute more complex query
sfc-gh-dprzybysz May 29, 2024
fe5177b
SNOW-979081: Read multiple columns and rows
sfc-gh-dprzybysz Jun 4, 2024
cdfe978
SNOW-979081: Support null
sfc-gh-dprzybysz Jun 4, 2024
c4068d7
SNOW-979081: Add loggin configuration (works only with files)
sfc-gh-dprzybysz Jun 4, 2024
eb55a7a
SNOW-979081: Use libsfclient logger instead of printf
sfc-gh-dprzybysz Jun 4, 2024
1df71b1
SNOW-979081: Terminate connection in a separate method
sfc-gh-dprzybysz Jun 4, 2024
7a072d6
SNOW-979081: Create generic module in driver source
sfc-gh-dprzybysz Jun 4, 2024
8897449
SNOW-979081: Configure dependencies for other operating systems
sfc-gh-dprzybysz Jun 4, 2024
6d2faf0
SNOW-979081: Add setting logging level
sfc-gh-dprzybysz Jun 5, 2024
caed6f8
SNOW-979081: Expose generic from snowflake module
sfc-gh-dprzybysz Jun 5, 2024
a0c944a
SNOW-979081: Read more connection properties
sfc-gh-dprzybysz Jun 5, 2024
75b90c6
SNOW-979081: Refactor parameter reading
sfc-gh-dprzybysz Jun 5, 2024
a089edf
SNOW-979081: Select result format
sfc-gh-dprzybysz Jun 5, 2024
9edbe52
SNOW-979081: Don't create string buffer
sfc-gh-dprzybysz Jun 5, 2024
56892a0
SNOW-979081: Stream result set
sfc-gh-dprzybysz Jun 5, 2024
9391164
SNOW-979081: Rename streaming to query without fetching rows
sfc-gh-dprzybysz Jun 6, 2024
7585f2d
SNOW-979081: Add streaming option with callback function
sfc-gh-dprzybysz Jun 6, 2024
c9cdec5
SNOW-979081: Fix namespace imports
sfc-gh-dprzybysz Jun 6, 2024
2602448
SNOW-979081: Add analogical nodejs examples for generic bindings
sfc-gh-dprzybysz Jun 6, 2024
1e63612
SNOW-979081: Don't create new context
sfc-gh-dprzybysz Jun 6, 2024
1b8392e
SNOW-979081: Upgrade darwin binaries to libsnowflakeclient 1.0.11
sfc-gh-dprzybysz Jun 6, 2024
60a4e44
SNOW-979081: Create variables for frequently reused elements
sfc-gh-dprzybysz Jun 6, 2024
9ea6810
SNOW-979081: Remove unnecessary includes
sfc-gh-dprzybysz Jun 7, 2024
5f2f33f
SNOW-979081: Add positional insert and select
sfc-gh-dprzybysz Jun 7, 2024
937afac
SNOW-979081: Prepare result format and connections before executing q…
sfc-gh-dprzybysz Jun 7, 2024
a3f2aea
SNOW-979081: Improve test cases for UTF-8 and binds
sfc-gh-dprzybysz Jun 10, 2024
8d6d011
SNOW-979081: Update to libsnowflakeclient 1.0.11
sfc-gh-dprzybysz Jun 10, 2024
19b4893
SNOW-979081: Simplify testing
sfc-gh-dprzybysz Jun 10, 2024
a600899
SNOW-979081: Fix some warnings
sfc-gh-dprzybysz Jun 10, 2024
91956b7
SNOW-979081: Fix linux build
sfc-gh-dprzybysz Jun 11, 2024
b1d3c34
SNOW-979081: Test async execution
sfc-gh-dprzybysz Jun 11, 2024
0f1a451
SNOW-979081: Remove unistd.h usage
sfc-gh-dprzybysz Jun 11, 2024
77d1c8b
SNOW-979081: Don't use new struct init syntax
sfc-gh-dprzybysz Jun 11, 2024
0cccf8d
SNOW-979081: Fix test builds
sfc-gh-dprzybysz Jun 11, 2024
27c2c93
SNOW-979081: Update aws linux libraries
sfc-gh-dprzybysz Jun 11, 2024
933e93c
SNOW-979081: Add libsfclient vs17 library and deps
sfc-gh-dprzybysz Jun 11, 2024
33542ad
SNOW-979081: Revert unnecessary changes in sf.js
sfc-gh-dprzybysz Jun 11, 2024
b49b010
SNOW-979081: Debug
sfc-gh-dprzybysz Jun 11, 2024
60f06cc
SNOW-979081: Reorder Linux dependencies
sfc-gh-dprzybysz Jun 12, 2024
a40cbcf
SNOW-979081: Set trace logging level to windows and linux builds
sfc-gh-dprzybysz Jun 12, 2024
f650eb3
SNOW-979081: Add cacert paths to test env variables
sfc-gh-dprzybysz Jun 12, 2024
e748908
SNOW-979081: Fix windows binds
sfc-gh-dprzybysz Jun 12, 2024
a09d2f1
SNOW-979081: Upgrade python to 3.11
sfc-gh-dprzybysz Jul 29, 2024
8ce4d14
SNOW-979081: Simplify perf test names format
sfc-gh-dprzybysz Jul 29, 2024
28f74d6
SNOW-979081: Run perf test N runs
sfc-gh-dprzybysz Jul 29, 2024
d7c53d4
SNOW-979081: Run perf test N runs dropping min and max
sfc-gh-dprzybysz Jul 29, 2024
a27bdc9
SNOW-979081: Add perf summary table
sfc-gh-dprzybysz Jul 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/build-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ jobs:
node-version: ${{ matrix.nodeVersion }}
- uses: actions/setup-python@v4
with:
python-version: '3.7'
python-version: '3.11'
- name: Download Build Artifacts
uses: actions/download-artifact@v3
with:
Expand All @@ -82,7 +82,8 @@ jobs:
test-windows:
needs: build
name: Tests on Windows
runs-on: windows-latest
runs-on: windows-2022
# runs-on: windows-2019 # we have binaries for vs16
strategy:
fail-fast: false
matrix:
Expand All @@ -95,7 +96,7 @@ jobs:
node-version: ${{ matrix.nodeVersion }}
- uses: actions/setup-python@v4
with:
python-version: '3.7'
python-version: '3.11'
architecture: 'x64'
- name: Download Build Artifacts
uses: actions/download-artifact@v3
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ wss-*-agent.config
wss-unified-agent.jar
whitesource/
.nyc_output
build
1 change: 1 addition & 0 deletions .npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ ci/
.pre-commit-config.yaml
Jenkinsfile
whitesource.config
build
120 changes: 120 additions & 0 deletions binding.gyp
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
{
"targets": [
{
"target_name": "generic_driver",
"sources": [
"bindings.cc"
],
"include_dirs": [
"libsfclient/include"
],
"libraries": [],
"conditions": [
[
"OS==\"mac\"",
{
"libraries": [
"<(module_root_dir)/libsfclient/lib/darwin/libsnowflakeclient.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/arrow/lib/libarrow.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/arrow_deps/lib/libjemalloc.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/arrow_deps/lib/libjemalloc_pic.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-auth.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-cal.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-common.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-compression.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-event-stream.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-http.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-io.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-mqtt.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-s3.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-c-sdkutils.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-checksums.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-cpp-sdk-core.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-cpp-sdk-s3.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/aws/lib/libaws-crt-cpp.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/azure/lib/libazure-storage-lite.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/boost/lib/libboost_atomic.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/boost/lib/libboost_filesystem.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/boost/lib/libboost_regex.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/boost/lib/libboost_system.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/curl/lib/libcurl.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/oob/lib/libtelemetry.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/openssl/lib/libcrypto.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/openssl/lib/libssl.a",
"<(module_root_dir)/libsfclient/deps-build/darwin/zlib/lib/libz.a"
]
}
],
[
"OS==\"win\"",
{
"libraries": [
"<(module_root_dir)/libsfclient/lib/win64/vs17/snowflakeclient.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/arrow/lib/arrow_static.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-auth.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-cal.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-common.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-compression.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-event-stream.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-http.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-io.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-mqtt.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-s3.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-c-sdkutils.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-checksums.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-cpp-sdk-core.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-cpp-sdk-s3.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/aws/lib/aws-crt-cpp.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/azure/lib/azure-storage-lite.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/boost/lib/libboost_atomic.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/boost/lib/libboost_filesystem.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/boost/lib/libboost_regex.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/boost/lib/libboost_system.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/curl/lib/libcurl_a.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/oob/lib/libtelemetry_a.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/openssl/lib/libcrypto_a.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/openssl/lib/libssl_a.lib",
"<(module_root_dir)/libsfclient/deps-build/win64/vs17/zlib/lib/zlib_a.lib"
]
}
],
[
"OS==\"linux\"",
{
"libraries": [
"<(module_root_dir)/libsfclient/lib/linux/libsnowflakeclient.a",
"<(module_root_dir)/libsfclient/deps-build/linux/arrow/lib/libarrow.a",
"<(module_root_dir)/libsfclient/deps-build/linux/arrow_deps/lib/libjemalloc_pic.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-cpp-sdk-s3.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-cpp-sdk-core.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-crt-cpp.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-event-stream.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-checksums.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-s3.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-auth.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-http.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-compression.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-io.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-cal.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-mqtt.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libs2n.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-common.a",
"<(module_root_dir)/libsfclient/deps-build/linux/aws/lib64/libaws-c-sdkutils.a",
"<(module_root_dir)/libsfclient/deps-build/linux/azure/lib/libazure-storage-lite.a",
"<(module_root_dir)/libsfclient/deps-build/linux/boost/lib/libboost_atomic.a",
"<(module_root_dir)/libsfclient/deps-build/linux/boost/lib/libboost_filesystem.a",
"<(module_root_dir)/libsfclient/deps-build/linux/boost/lib/libboost_regex.a",
"<(module_root_dir)/libsfclient/deps-build/linux/boost/lib/libboost_system.a",
"<(module_root_dir)/libsfclient/deps-build/linux/curl/lib/libcurl.a",
"<(module_root_dir)/libsfclient/deps-build/linux/oob/lib/libtelemetry.a",
"<(module_root_dir)/libsfclient/deps-build/linux/openssl/lib/libcrypto.a",
"<(module_root_dir)/libsfclient/deps-build/linux/openssl/lib/libssl.a",
"<(module_root_dir)/libsfclient/deps-build/linux/uuid/lib/libuuid.a",
"<(module_root_dir)/libsfclient/deps-build/linux/zlib/lib/libz.a"
]
}
]
]
}
]
}
Loading
Loading