Skip to content

External API

Jump to bottom
Erik McClure edited this page Dec 1, 2019 · 7 revisions

The inNative External API is what applications call to assemble an environment and compile WebAssembly modules. The functions covered here are acquired using a call to innative_runtime, which resolves all the function pointers in an INExports struct, either statically or dynamically. This ensures that switching between embedded and dynamic runtimes does not require any code changes. In addition to these functions, there is also the Metadata API and the Schema Manipulation API. The second part of this page covers the data structures used in the API functions.

External Functions

Environment* CreateEnvironment(unsigned int modules, unsigned int maxthreads, const char* arg0);
  • modules: The number of modules that are expected to be added to the environment - can just be an estimate.
  • maxthreads: The maximum number of threads to use for multithreading. 0 defaults to logical cores.
  • arg0: The first argument sent to the program. Used to determine binary location on POSIX systems.
  • Returns a pointer to a new Environment structure, which must be destroyed using DestroyEnvironment(). Creates an environment initialized to default values, which must be destroyed using DestroyEnvironment().
void AddModule(Environment* env, const void* data, uint64_t size, const char* name, int* err);
  • env: The environment to modify.
  • data: Either a pointer to the module in memory, or a UTF8 encoded null-terminated string pointing to a file that contains the module.
  • size: The length of the memory that the data pointer points to, or zero if the data pointer is actually a UTF8 encoded null terminated string.
  • name: A name to use for the module. If the module data does not contain a name, this will be used.
  • err: A pointer to an integer that receives an error code should the function fail. Not valid until FinalizeEnvironment() is called.
  • Returns Nothing

Adds a module to an environment. This could happen asynchronously if multithreading is enabled, in which case err won't be valid until FinalizeEnvironment() is called to resolve all pending module loads.

int AddModuleObject(Environment* env, const Module* m);
  • env The environment to modify.
  • m The module to add to the environment. This must be a valid Module object or the validation step will fail. The module will be copied into the environment - further modifications to the given module pointer won't affect the environment's internal copy.
  • Returns a standard IN_ERROR code, or ERR_SUCCESS if it succeeds.

Adds a prebuilt module object to the environment. This happens synchronously, regardless of multithreading flags.

void AddWhitelist (Environment* env, const char* module_name, const char* export_name);
  • env: The environment to modify.
  • module_name: The name of a module, in case the C function is actually a name-mangled webassembly function. This parameter should be null for standard C functions.
  • export_name: The name of the function to add to the whitelist. Must be a valid UTF8 webassembly function name.
  • Returns nothing

Adds a whitelist entry to the environment. This will only be used if the whitelist is enabled via the ENV_WHITELIST flag.

enum IN_ERROR AddEmbedding(Environment* env, int tag, const void* data, uint64_t size);
  • env: The environment to modify.
  • tag: An IN_EMBEDDING_TAGS enumeration option that provides an optional hint for the type of file. To automatically choose, set to 0 (equivalent to IN_TAG_ANY).
  • data: Either a pointer to memory, or a UTF8 encoded null-terminated string pointing to a file.
  • size: The length of the memory that the data pointer points to. If size is 0, the data pointer is actually a null terminated UTF8 encoded file path.
  • Returns a standard IN_ERROR code, or ERR_SUCCESS if it succeeds.

Adds an embedding to the environment. This is usually a static or shared C library that exposes C functions that the webassembly modules can call.

enum IN_ERROR AddCustomExport(Environment* env, const char* symbol);
  • env: The environment to modify.
  • symbol: The null-terminated name of the symbol to export.
  • Returns a standard IN_ERROR code, or ERR_SUCCESS if it succeeds.

Tells the linker to export the given symbol from the resulting binary.

enum IN_ERROR FinalizeEnvironment(Environment* env);
  • env: The environment to finalize
  • Returns a standard IN_ERROR code, or ERR_SUCCESS if it succeeds. Finalizes the environment, blocking until all modules have finished loading (in case of any asynchronous loads) and ensures all configuration data is loaded.
enum IN_ERROR Validate(Environment* env);
  • env: The environment to verify.
  • Returns a standard IN_ERROR code, or ERR_SUCCESS if it succeeds.

Validates all the modules in the environment using the current configuration.

enum IN_ERROR Compile(Environment* env, const char* file);
  • env: The environment to compile.
  • file: The path of the output file that is produced.
  • Returns a standard IN_ERROR code, or ERR_SUCCESS if it succeeds.

Compiles all the modules in the environment using the current configuration and any cached results into a binary file.

void* LoadAssembly(const char* file);
  • file: the path of the file to load.
  • Returns a pointer to the loaded assembly that must be freed using FreeAssembly(), or NULL if the load fails.

Loads a webassembly binary (usually a dynamic library) produced by Compile into memory, allowing you to load functions and other exported symbols.

void FreeAssembly (void* assembly);
  • assembly: A pointer to a webassembly binary loaded by LoadAssembly().
  • Returns nothing.

Frees a webassembly binary, unloading it from memory and performing any cleanup required.

void ClearEnvironmentCache(Environment* env, Module* m);
  • env: The environment to clear.
  • m: An optional module whose compilation cache will be cleared. If this is a null pointer, instead clears the entire cache of the environment.
  • Returns nothing.

Clears the environment's cached compilation of a given module, or clears the entire cache if the module is a null pointer.

const char* GetTypeEncodingString(int type_encoding);
  • error_code: The TYPE_ENCODING value to get the string representation of.
  • Returns a string representation of the type encoding, or NULL if it fails.

Returns the string representation of a TYPE_ENCODING enumeration, or NULL if the lookup fails. Useful for debuggers.

const char* GetErrorString(int error_code);
  • error_code: The IN_ERROR code to get the string representation of.
  • Returns a string representation of the error code, or NULL if it fails.

Returns the string representation of an IN_ERROR enumeration, or NULL if the lookup fails. Useful for debuggers.

void DestroyEnvironment(Environment* env);
  • env: The environment to destroy.
  • Returns nothing.

Destroys an environment and safely deconstructs all it's caches and memory allocations.

int CompileScript(const uint8_t* data, size_t sz, Environment* env, bool always_compile, const char* output);
  • data: Either a pointer to the script in memory, or a UTF8 encoded null-terminated string pointing to a file that contains the script.
  • sz: The length of the memory that the data pointer points to, or zero if the data pointer is actually a UTF8 encoded null terminated string.
  • env: The environment to execute the script with. This environment can potentially be modified by the script being executed.
  • always_compile: By default, this function only performs a compilation when necessary to call a function. If this is set to true, all modules are compiled even if no function inside them is called.
  • output: Sets the output directory where compilation results should be stored. Intermediate results will still be in env->objpath
  • Returns an IN_ERROR error code as an integer, or ERR_SUCCESS (0) if it succeeds.

Compiles and executes a .wast script using the given environment. This execution will modify the environment and add all modules referenced in the script according to the registration rules.

Static Functions

These functions are not part of the official External API, but can be accessed when statically linking or embedding the runtime. They are mostly used by inNative tools for specialized operations that most normal programs don't need to worry about.

IN_COMPILER_DLLEXPORT extern void innative_set_work_dir_to_bin(const char* arg0);
  • arg:0 the first argument sent to the program. Used to determine binary location on POSIX systems.
  • Returns nothing

Convenience function that sets the working directory to the executable's root directory.

IN_COMPILER_DLLEXPORT extern int innative_install(const char* arg0, bool full);
  • arg:0 the first argument sent to the program. Used to determine binary location on POSIX systems.
  • full: On windows, performs a "full" installation, which associates the runtime with .wat/.wast/.wasm files.
  • Returns an IN_ERROR error code as an integer, or ERR_SUCCESS (0) if it succeeds.

Installs this runtime, usually only called by innative-cmd, which makes assumptions about file locations.

IN_COMPILER_DLLEXPORT extern int innative_uninstall();
  • Returns an IN_ERROR error code as an integer, or ERR_SUCCESS (0) if it succeeds.

Uninstalls whatever runtime version this is from the operating system.

IN_COMPILER_DLLEXPORT extern int innative_compile_llvm(const char** files, size_t n, int flags, const char* out, FILE* log);
  • files: An array of UTF8 file paths to compile.
  • n: The length of the 'files' array.
  • flags: Environment flags to compile with. This is mostly used to set the ENV_LIBRARY flag in case this should be a shared library.
  • out: The output file that will store the compilation result.
  • log: A C FILE* stream that should be used for logging errors or warnings.
  • Returns an IN_ERROR error code as an integer, or ERR_SUCCESS (0) if it succeeds.

Performs a reverse compilation of LLVM IR into WebAssembly using the built-in LLVM version in this runtime.

Data Structures

This is a list of the external data structures used by the API, with instructions on how they work.

typedef struct __WASM_ENVIRONMENT
{
  size_t n_modules;
  size_t size; 
  size_t capacity; 
  Module* modules; 
  Embedding* embeddings; 
  ValidationError* errors; 
  uint64_t flags; 
  uint64_t features; 
  uint64_t optimize; 
  unsigned int maxthreads;
  const char* libpath;
  const char* objpath;
  const char* linker;
  const char* system;
  struct __WASM_ALLOCATOR* alloc;
  int loglevel; 
  FILE* log; 
  void(*wasthook)(void*); 

  struct kh_modules_s* modulemap;
  struct kh_modulepair_s* whitelist;
  struct kh_cimport_s* cimports;
  __LLVM_CONTEXT* context;
} Environment;

The Environment struct represents the compiler's current state, and holds all current modules, embeddings, flags, and any other configuration data. When compiling a binary, this holds all relevent information the compiler needs (except the output file path). Below, we describe the public fields of the Environment struct. Any fields not covered here are internal and should not be touched by other programs.

size_t n_modules

This represents the number of fully loaded modules, which is seperate from the total number of modules. This is simply a count used by FinalizeEnvironment() to figure out when all asyncronous loads have finished. Should be treated as read-only, but could be used to feed a progress bar if accessed using atomic loads.

size_t size

Total number of modules that are currently loading or have been loaded. This is the real number of modules that have been added to the Environment. Once size is equal to n_modules, they have all been loaded. Should be treated as read-only, can be used to tell the maximum possible value of n_modules for a progress bar.

size_t capacity

Internal capacity of the modules array. Do not use.

Module* modules

Pointer to the modules array. This is safe to read and modify after FinalizeEnvironment() has been called. Both n_modules and size can be used to retrieve the array length, but n_modules is preferred.

Embedding* embeddings

This is a singly linked list of embedding environments. It should be treated as read-only, but it can be walked by the host. Use AddEmbedding to correctly append embeddings to this list.

ValidationError* errors

This is a singly linked list of either validation errors or certain compiler errors. It should be considered read-only, but it is always safe to iterate through, even asyncronously during compilation (errors are added on the end using a lockless, thread-safe algorithm).

uint64_t flags

Stores standard flags from the WASM_ENVIRONMENT_FLAGS enumeration.

uint64_t features

Stores feature flags from the WASM_FEATURE_FLAGS enumeration.

uint64_t optimize

Stores optimization flags from the WASM_OPTIMIZE enumeration.

unsigned int maxthreads

Stores the maximum number of threads to use for multithreaded compilation. Set when creating the environment and shouldn't be changed afterwards.

const char* libpath

This is the directory used to find library files, including the Default Environment. Defaults to the EXE's current directory, and can be changed at any time, but changing it may invalidate cached compilation results. On Linux, /usr/lib/ is always searched in addition to this directory.

const char* objpath

Directory used to store intermediate compilation results, like .o files. These are only destroyed once the Environment is destroyed, so if you are compiling multiple things, the Environment can re-use cached compilation results when appropriate. Defaults to NULL, and if it is still set to NULL when compilation starts, will be set to the output file's directory. In order to preserve intermediate results, it will stay set to the first output file's directory. If you are compiling more than one thing, it is highly recommended you actually set this to a suitable directory before starting. It can be changed at any time, except during compilation, but doing so will cause inNative to lose track of any existing intermediate results and fail to delete them.

const char* linker

Species an alternative linker, usually the system linker. This linker must use the operating system's standard linker interface, or it simply won't work. Intended as a fallback in case a bug in LLD prevents it from linking correctly. Can be changed at any time, but may invalidate cached compilation results.

const char* system

Determines the module that is considered the "system level" module from which to import C functions. This module is basically aliased to a blank module name for the purposes of Name Mangling. Can be changed at any point, but may invalidate or break compilation completely if done during or in-between compilation.

int loglevel

Determines the log level, which should be set to a value from WASM_LOG_LEVEL. Only messages with priority equal to or greater than the given level will be logged. Defaults to LOG_WARNING, which logs all warnings, errors, and fatal errors. Setting it to LOG_NOTICE will log all notices, warnings, errors, and fatal errors, which is useful for debugging WebAssembly modules. LOG_DEBUG is generally for debugging inNative itself

FILE* log

Determines the output stream used for logging purposes. Can be changed in-between operations, but usually should be set after environment creation and never touched afterwards.

void(*wasthook)(void*)

This is a hook function that fires whenever the .wast parser performs a compilation. Used primarily for progress bars or debugging purposes.

Clone this wiki locally