Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not able to use exported c function as custom function in query #310

Open
AadhiKat opened this issue Nov 26, 2019 · 4 comments
Open

Not able to use exported c function as custom function in query #310

AadhiKat opened this issue Nov 26, 2019 · 4 comments
Labels

Comments

@AadhiKat
Copy link

AadhiKat commented Nov 26, 2019

I wrote a c function and compiled using the command

emcc lib/alternate_case.c -s WASM=1 -s EXPORTED_FUNCTIONS="['_alternatecase']" -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' -s ASSERTIONS=1 -o public/alternate_case.js

Here is my C function

#include<stdio.h>
#include<string.h>
#include<ctype.h>
#include<emscripten.h>

int main() {
  return 1;
}

char * alternatecase(char *str) {
    char *initial_ptr;
    initial_ptr = str;
    for (int i=0; i < *str!='\0' ; i++){
        if( (i % 2) == 0)
            *str = tolower(*str);
        else
            *str = toupper(*str);
        str++;
    }
    return initial_ptr;
}

Index.html

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>My first Web Assembly Demo</title>
  </head>
  <body>
    <h1> MY first web assembly demo</h1>
    Output is in Javascript console
    <main>
      <button id="execute" class="button">Execute</button>
      <pre id="output"> Results will be displayed here </pre>
    </main>
  </body>
  <script src='/dist/sql-wasm.js'></script>
  <script src='alternate_case.js'></script>
  <script type="text/javascript" src="exporting-c-function.js"></script>
</html>

I took the sql-wasm.js from this repository. alternate_case.js is the compiled file from emcc.
This is the exporting-c-function.js

config = {
      locateFile: filename => `dist/sql-wasm.wasm`
    }

    initSqlJs(config).then(function(SQL){
      //Create the database
      var db = new SQL.Database();
      // Run a query without reading the results

      const func = cwrap('alternatecase','string', ['string']);
      db.create_function("altcase" , func)

      sqlstr = "CREATE TABLE hello (a int, b char);";
      sqlstr += "INSERT INTO hello VALUES (0, 'hello');"
      sqlstr += "INSERT INTO hello VALUES (1, 'world');"
      db.run(sqlstr); // Run the query without returning anything

      //db.run("INSERT INTO hello VALUES (alternatecase('awdqwdq'));"); // Inserts 10 and 'Hello world'
       db.run('SELECT altcase(b) from hello;');
});

It is throwing this error in the console
Error: wrong number of arguments to function altcase()

And it accepts only 0 arguments.
Note: if I try from browser console , I am able to use the function and get the output.
How to use this as custom function?
Also how to create custom function in web workers and use it ?

@lovasoa
Copy link
Member

lovasoa commented Nov 26, 2019

If you want to define your own function in C, you should probably compile it together with SQLite as a single wasm file, not as a separate file. You should read both the SQLite C API documentation and the emscripten documentation.

Alternatively, you can define your function directly in javascript, and use it from sql.js:

  // You can also use JavaScript functions inside your SQL code
  // Create the js function you need
  function add(a, b) {return a+b;}
  // Specifies the SQL function's name, the number of it's arguments, and the js function to use
  db.create_function("add_js", add);
  // Run a query in which the function is used
  db.run("INSERT INTO hello VALUES (add_js(7, 3), add_js('Hello ', 'world'));"); // Inserts 10 and 'Hello world'

@AadhiKat
Copy link
Author

Thanks! Can you provide any information on where should I start using this repo ? As I see , the exported_functions is taken from the sqlite url. Will there be performance difference if those udfs are created in c/c++ instead of js ?

My requirement would be to store around 5 mb of data in a table and then perform some queries.
All kinds of actions that can be performed in spark sql. Altering table , deleting columns and such.
Also would like to know if the 5mb data can be persisted in the browser after each transformations?
The operations could range from medium to more complex queries.
Also will there be browser lag on loading this js and performing these operations ??

@lovasoa
Copy link
Member

lovasoa commented Nov 26, 2019

Will there be performance difference if those udfs are created in c/c++ instead of js ?

Certainly. But you should run tests to know whether they are significant or not. Please report your findings here !

Also would like to know if the 5mb data can be persisted in the browser after each transformations?

You can use IndexedDB for that. But note that the storage space is limited

https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Client-side_web_APIs/Client-side_storage#Storing_simple_data_%E2%80%94_web_storage

Also will there be browser lag on loading this js and performing these operations ??

You can perform the heavy operations in a web worker in order not to block the UI thread. This should avoid lags.

https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers

@brodycj
Copy link
Contributor

brodycj commented Feb 18, 2020

I started work in PR #320 to show how it would be possible to embed non-standard C functions. In short, you would do sqlite3_create_function_v2 to add each custom C function whenever you open the database. You can see more concretely here: https://github.com/brodybits/sql.js/tree/custom-eu-string-functions

https://github.com/kripken/sql.js/issues/310#issuecomment-558533849 shows how you can do this with a custom function implemented in JavaScript.

I think it would be nice if we can get this information documented (someday).

@brodycj brodycj mentioned this issue Feb 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants