Caused by: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- embulk/output/bigquery at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)

[nitishrjr]

I am trying to load data into bigquery. However I am getting the below error continuously. I have tried multiple times including install/uninstall.

2023-09-28 05:59:36.733 +0000 [INFO] (0001:transaction): Loaded JRuby runtime 9.4.3.0
2023-09-28 05:59:36.824 +0000 [INFO] (0001:transaction): Loaded plugin embulk-input-sqlserver (0.13.2)
2023-09-28 05:59:36.988 +0000 [INFO] (0001:transaction): Embulk system property "plugins.output.bigquery" is not set.
2023-09-28 05:59:36.989 +0000 [INFO] (0001:transaction): Embulk system property "plugins.default.output.bigquery" is not set.
org.embulk.config.ConfigException: OutputPlugin 'bigquery' is not found.
org.jruby.exceptions.LoadError: (LoadError) no such file to load -- google/apis/bigquery_v2
at org.embulk.plugin.PluginManager.buildPluginNotFoundException(PluginManager.java:155)
at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:99)
at org.embulk.spi.ExecSessionInternal.newPlugin(ExecSessionInternal.java:358)
at org.embulk.spi.ExecInternal.newPlugin(ExecInternal.java:50)
at org.embulk.exec.BulkLoader$ProcessPluginSet.(BulkLoader.java:442)
at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:516)
at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:36)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:353)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:350)
at org.embulk.spi.ExecInternal.doWith(ExecInternal.java:26)
at org.embulk.exec.BulkLoader.run(BulkLoader.java:350)
at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:278)
at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:288)
at org.embulk.EmbulkRunner.run(EmbulkRunner.java:153)
at org.embulk.cli.EmbulkRun.runInternal(EmbulkRun.java:108)
at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:24)
at org.embulk.cli.Main.main(Main.java:53)
Suppressed: org.embulk.plugin.PluginSourceNotMatchException
at org.embulk.plugin.BuiltinPluginSource.newPlugin(BuiltinPluginSource.java:244)
at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:76)
... 15 more
Suppressed: org.embulk.plugin.PluginSourceNotMatchException
at org.embulk.plugin.maven.MavenPluginRegistry.findPluginPaths(MavenPluginRegistry.java:143)
at org.embulk.plugin.maven.MavenPluginRegistry.lookup(MavenPluginRegistry.java:63)
at org.embulk.plugin.maven.MavenPluginSource.newPlugin(MavenPluginSource.java:33)
at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:82)
... 15 more
Suppressed: org.embulk.plugin.PluginSourceNotMatchException
at org.embulk.plugin.SelfContainedPluginRegistry.search(SelfContainedPluginRegistry.java:77)
at org.embulk.plugin.SelfContainedPluginRegistry.lookup(SelfContainedPluginRegistry.java:64)
at org.embulk.plugin.SelfContainedPluginSource.newPlugin(SelfContainedPluginSource.java:44)
at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:88)
... 15 more
Suppressed: org.embulk.plugin.PluginSourceNotMatchException: Failed to initialize embulk-output-bigquery.
at org.embulk.jruby.JRubyPluginSource.newPlugin(JRubyPluginSource.java:89)
at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:94)
... 15 more
Caused by: org.jruby.embed.InvokeFailedException: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- google/apis/bigquery_v2
at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.doInvokeMethod(EmbedRubyObjectAdapterImpl.java:251)
at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:156)
at org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1448)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.embulk.jruby.ScriptingContainerDelegateImpl.callMethod(ScriptingContainerDelegateImpl.java:644)
at org.embulk.jruby.LazyScriptingContainerDelegate.callMethod(LazyScriptingContainerDelegate.java:230)
at org.embulk.jruby.JRubyPluginSource.newPlugin(JRubyPluginSource.java:87)
... 16 more
Caused by: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- google/apis/bigquery_v2
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
at uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.rubygems.core_ext.kernel_require.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:160)
at RUBY.

(/home/azureuser/.embulk/lib/gems/gems/embulk-output-bigquery-0.6.9/lib/embulk/output/bigquery/bigquery_client.rb:1)
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
at org.jruby.RubyKernel.require_relative(org/jruby/RubyKernel.java:1098)
at RUBY.(/home/azureuser/.embulk/lib/gems/gems/embulk-output-bigquery-0.6.9/lib/embulk/output/bigquery.rb:6)
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
at uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.rubygems.core_ext.kernel_require.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:160)
at RUBY.require_and_show(/home/azureuser/.embulk/lib/gems/gems/embulk-0.11.0-java/lib/embulk/plugin_registry.rb:81)
at RUBY.search(/home/azureuser/.embulk/lib/gems/gems/embulk-0.11.0-java/lib/embulk/plugin_registry.rb:39)
at RUBY.lookup(/home/azureuser/.embulk/lib/gems/gems/embulk-0.11.0-java/lib/embulk/plugin_registry.rb:26)
at RUBY.lookup(/home/azureuser/.embulk/lib/gems/gems/embulk-0.11.0-java/lib/embulk/plugin.rb:197)
at RUBY.new_java_output(/home/azureuser/.embulk/lib/gems/gems/embulk-0.11.0-java/lib/embulk/plugin.rb:159)
at RUBY.new_java_output(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/forwardable.rb:238)
Caused by: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- google/apis/bigquery_v2
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
at uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.rubygems.core_ext.kernel_require.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:148)
... 12 more
Caused by: org.jruby.exceptions.LoadError: (LoadError) no such file to load -- embulk/output/bigquery
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
at uri_3a_classloader_3a_.META_minus_INF.jruby_dot_home.lib.ruby.stdlib.rubygems.core_ext.kernel_require.require(uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:85)
... 6 more

Error: OutputPlugin 'bigquery' is not found.
org.jruby.exceptions.LoadError: (LoadError) no such file to load -- google/apis/bigquery_v2

---

if anyone faced the same issue, could you please tell me how did you resolve it.

[hiroyuki-sato]

Hello, @nitishrjr

If you want to use Embulk v0.11.x, You need to wait for a new embulk-output-bigquery version.
(Or build a gem by yourself. If you are interested in this, I'll post how to do it. I'm looking for someone help for test.)
embulk/embulk-output-bigquery#155

As a workaround, You can use Embulk v0.9.

Create Gemfile

source "https://rubygems.org"

gem "jwt", "= 2.3.0"
gem "public_suffix", "= 4.0.7"
gem "multipart-post", "= 2.1.1"
gem "jruby-openssl", "= 0.10.7"
gem "embulk-output-bigquery", "= 0.6.9"

And execute embulk bundle install.

There are some reasons. If you want to know the detail please let me know.

Reference

• 'embulk gem install' may fail in Embulk v0.9: incompatibility with RubyGems
• embulk→bigqueryのメモ (Japanese contents)

[dmikurube]

We see no response for a year. Let me close this discussion.