A native Lua implementation of Spark SQL.
This is a companion module for Stuart, the Spark runtime for embedding and edge computing.
$ luarocks install stuart-sqlIf you are using Lua < 5.3 and also want to work with Parquet files, then:
$ luarocks install parquetlocal lodash = require 'lodash'
local SparkSession = require 'stuart-sql.SparkSession'
local session = SparkSession.builder():getOrCreate()
local centroidsDataFrame = session.read:parquet('my-kmeans-model/part3.parquet')
local centroids = centroidsDataFrame:rdd():collect()
lodash.print('centroids=', centroids)
centroids= {{0, {3,4,5}}} -- rowid, values$ busted -v
●●
3 successes / 0 failures / 0 errors / 0 pending : 0.063098 seconds$ docker build -f Test-Lua52.Dockerfile -t test .
$ docker run -it test busted -v
●●
3 successes / 0 failures / 0 errors / 0 pending : 0.063098 seconds