api.md

Fennel's Lua API

The fennel module provides the following functions for use when embedding Fennel in a Lua program. If you're writing a pure Fennel program or working on a system that already has Fennel support, you probably don't need this.

Any time a function takes an options table argument, that table will usually accept these fields:

• allowedGlobals: a sequential table of strings of the names of globals which the compiler will allow references to. Set to false to disable checks.
• correlate: when this is truthy, Fennel attempts to emit Lua where the line numbers match up with the Fennel input code; useful for situation where code that isn't under your control will print the stack traces.
• useMetadata (since 0.3.0): enables or disables metadata, allowing use of the doc macro. Intended for development purposes (see performance note); defaults to true for REPL only.
• requireAsInclude (since 0.3.0): Alias any static require calls to the include special, embedding the module code inline in the compiled output. If the module name isn't a string literal or resolvable at compile time, falls back to require at runtime. Can be used to embed both fennel and Lua modules.
• env: an environment table in which to run the code; see the Lua manual.
• compilerEnv: an environment table in which to run compiler-scoped code for macro definitions and eval-compiler calls. Internal Fennel functions such as list, sym, etc. will be exposed in addition to this table. Defaults to a table containing limited known-safe globals. Pass _G to disable sandboxing.
• unfriendly: disable friendly compiler/parser error messages.

You can pass the string "_COMPILER" as the value for env; it will cause the code to be run/compiled in a context which has all compiler-scoped values available. This can be useful for macro modules or compiler plugins.

Note that only the fennel module is part of the public API. The other modules (fennel.utils, fennel.compiler, etc) should be considered compiler internals subject to change.

Start a configurable repl

fennel.repl([options])

Takes these additional options:

• readChunk(): a function that when called, returns a string of source code. The empty is string is used as the end of source marker.
• pp: a pretty-printer function to apply on values.
• onValues(values): a function that will be called on all returned top level values.
• onError(errType, err, luaSource): a function that will be called on each error. errType is a string with the type of error, can be either, 'parse', 'compile', 'runtime', or 'lua'. err is the error message, and luaSource is the source of the generated lua code.

src/fennel/view.fnl will produce output that can be fed back into Fennel (other than functions, coroutines, etc) but you can use a 3rd-party pretty-printer that produces output in Lua format if you prefer.

If you don't provide allowedGlobals then it defaults to being all the globals in the environment under which the code will run. Passing in false here will disable global checking entirely.

By default, metadata will be enabled and you can view function signatures and docstrings with the doc macro from the REPL.

Evaulate a string of Fennel

local result = fennel.eval(str[, options[, ...]])

The options table may also contain:

• filename: override the filename that Lua thinks the code came from.

Additional arguments beyond options are passed to the code and available as ....

Evaluate a file of Fennel

local result = fennel.dofile(filename[, options[, ...]])

Use Lua's built-in require function

table.insert(package.loaders or package.searchers, fennel.searcher)
local mylib = require("mylib") -- will compile and load code in mylib.fnl

Normally Lua's require function only loads modules written in Lua, but you can install fennel.searcher into package.searchers (or in Lua 5.1 package.loaders) to teach it how to load Fennel code.

If you would rather change some of the options you can use fennel.makeSearcher(options) to get a searcher function that's equivalent to fennel.searcher but overrides the default options table.

The require function is different from fennel.dofile in that it searches the directories in fennel.path for .fnl files matching the module name, and also in that it caches the loaded value to return on subsequent calls, while fennel.dofile will reload each time. The behavior of fennel.path mirrors that of Lua's package.path.

If you install Fennel into package.searchers then you can use the 3rd-party lume.hotswap function to reload modules that have been loaded with require.

Get Fennel-aware stack traces.

The fennel.traceback function works like Lua's debug.traceback function, except it tracks line numbers from Fennel code correctly.

If you are working on an application written in Fennel, you can override the default traceback function to replace it with Fennel's:

debug.traceback = fennel.traceback

Search the path for a module without loading it

print(fennel.searchModule("my.mod", package.path))

If you just want to find the file path that a module would resolve to without actually loading it, you can use fennel.searchModule. The first argument is the module name, and the second argument is the path string to search. If none is provided, it defaults to Fennel's own path.

Returns nil if the module is not found on the path.

Compile a string into Lua (can throw errors)

local lua = fennel.compileString(str[, options])

Accepts indent as a string in options causing output to be indented using that string, which should contain only whitespace if provided. Unlike the other functions, the compile functions default to performing no global checks, though you can pass in an allowedGlobals table in options to enable it.

Compile an iterator of bytes into a string of Lua (can throw errors)

local lua = fennel.compileStream(strm[, options])

Accepts indent in options as per above.

Compile a data structure (AST) into Lua source code (can throw errors)

The code can be loaded via dostring or other methods. Will error on bad input.

local lua = fennel.compile(ast[, options])

Accepts indent in options as per above.

Get an iterator over the bytes in a string

local stream = fennel.stringStream(str)

Converts an iterator for strings into an iterator over their bytes

Useful for the REPL or reading files in chunks. This will NOT insert newlines or other whitespace between chunks, so be careful when using with io.read(). Returns a second function, clearstream, which will clear the current buffered chunk when called. Useful for implementing a repl.

local bytestream, clearstream = fennel.granulate(chunks)

Converts a stream of bytes to a stream of AST nodes

The fennel.parser function returns a stateful iterator function. It returns true in the first return value if an AST was read, and returns nil if an end of file was reached without error. Will error on bad input or unexpected end of source.

local parse = fennel.parser(strm)
local ok, ast = parse()

-- Or use in a for loop
for ok, ast in parse do
    print(ok, ast)
end

The fennel.parser function takes two optional arguments; a filename and a table of options. Supported options are both booleans that default to false:

• unfriendly: disable enhanced parse error reporting
• comments: include comment nodes in AST

AST node definition

The AST returned by the parser consists of data structures representing the code. Passing AST nodes to the fennel.view function will give you a string which should round-trip thru the parser to give you the same data back. The same is true with tostring, except it does not work with kv tables.

AST nodes can be any of these types:

list

A list represents a call to function/macro, or destructuring multiple return values in a binding context. It's represented as a table which can be identified using the fennel.list? predicate function or constructed using fennel.list which takes any number of arguments for the contents of the list.

The list also contains these keys indicating where it was defined: filename, line, bytestart, and byteend. This data is used for stack traces and for pinpointing compiler error messages.

sequence/kv table

These are table literals in Fennel code produced by square brackets (sequences) or curly brackets (kv tables). Sequences can be identified using the fennel.sequence? function and constructed using fennel.sequence. There is no predicate or constructor for kv tables; any table which is not one of the other types is assumed to be one of these.

At runtime there is no difference between sequences and kv tables which use monotonically increasing integer keys, but the parser is able to distinguish between them.

Sequences have their source data in filename, line, etc keys just like lists. But kv tables cannot have this, because adding arbitrary keys to the table would change its contents, so these fields are instead stored on the metatable. The metatable for kv tables also includes a keys sequence which tells you which order the keys appeared originally, since kv tables are unordered and there would otherwise be no way to reconstruct this information.

symbol

Symbols typically represent identifiers in Fennel code. Symbols can be identified with fennel.sym? and constructed with fennel.sym which takes a string name as its first argument and a source data table as the second. Symbols are represented as tables which store their source data in fields on themselves. Unlike the other tables in the AST, they do not represent collections; they are used as scalar types.

Note: nil is not a valid AST; code that references nil will have the symbol named "nil" which unfortunately prints in a way that is visually indistinguishable from actual nil.

vararg

This is a special type of symbol-like construct (...) indicating functions using a variable number of arguments. Its meaning is the same as in Lua. It's identified with fennel.varg? and constructed with fennel.varg.

number/string/boolean

These are literal types defined by Lua. They cannot carry source data.

comment

By default, ASTs will omit comments. However, when the :comment field is set in the parser options, comments will be included in the parsed values. They are identified using fennel.comment? and constructed using the fennel.comment function. They are represented as tables that have source data as fields inside them.

In most data context, comments just get included inline in a list or sequence. However, in a kv table, this cannot be done, because kv tables must have balanced key/value pairs, and including comments inline would imbalance these or cause keys to be considered as values and vice versa. So the comments are stored on the comments field of metatable instead, keyed by the key or value they were attached to.

Work with docstrings and metadata

(Since 0.3.0)

When running a REPL or using compile/eval with metadata enabled, each function declared with fn or λ/lambda will use the created function as a key on fennel.metadata to store the function's arglist and (if provided) docstring. The metadata table is weakly-referenced by key, so each function's metadata will be garbage collected along with the function itself.

You can work with the API to view or modify this metadata yourself, or use the doc macro from fennel to view function documentation.

In addition to direct access to the metadata tables, you can use the following methods:

• fennel.metadata:get(func, key): get a value from a function's metadata
• fennel.metadata:set(func, key, val): set a metadata value
• fennel.metadata:setall(func, key1, val1, key2, val2, ...): set pairs
• fennel.doc(func, fnName): print formatted documentation for function using name. Utilized by the doc macro, name is whatever symbol you operate on that's bound to the function.

greet = fennel.eval([[
(λ greet [name] "Say hello" (print (string.format "Hello, %s!" name)))
]], {useMetadata = true})

-- fennel.metadata[greet]
-- > {"fnl/docstring" = "Say hello", "fnl/arglist" = ["name"]}

-- works because greet was set globally above for example purposes only
fennel.eval("(doc greet)", { useMetadata = true })
-- > (greet name)
-- >   Say hello

fennel.metadata:set(greet, "fnl/docstring", "Say hello!!!")
fennel.doc(greet, "greet!")
--> (greet! name)
-->   Say hello!!!

Metadata performance note

Enabling metadata in the compiler/eval/REPL will cause every function to store a new table containing the function's arglist and docstring in the metadata table, weakly referenced by the function itself as a key.

This may have a performance impact in some applications due to the extra allocations and garbage collection associated with dynamic function creation. The impact hasn't been benchmarked, and may be minimal particularly in luajit, but enabling metadata is currently recommended for development purposes only to minimize overhead.

Load Lua code in a portable way

This isn't Fennel-specific, but the loadCode function takes a string of Lua code along with an optional environment table and filename string, and returns a function for the loaded code which will run inside that environment, in a way that's portable across any Lua 5.1+ version.

local f = fennel.loadCode(luaCode, { x = y }, "myfile.lua")

Plugins

Fennel's plugin system is extremely experimental and exposes internals of the compiler in ways that no other part of the compiler does. It should be considered unstable; changes to the compiler in future versions are likely to break plugins, and each plugin should only be assumed to work with specific versions of the compiler that they're tested against. The backwards-compatibility guarantees of the rest of Fennel do not apply to plugins.

Compiler plugins allow the functionality of the compiler to be extended in various ways. A plugin is a module containing various functions in fields named after different compiler extension points. When the compiler hits an extension point, it will call each plugin's function for that extension point, if provided, with various arguments; usually the AST in question and the scope table.

• symbol-to-expression
• call
• do
• fn
• destructure

The destructure extension point is different because instead of just taking ast and scope it takes a from which is the AST for the value being destructured and a to AST which is the AST for the form being destructured to. This is most commonly a symbol but can be a list or a table.

The scope argument is a table containing all the compiler's information about the current scope. Most of the tables here look up values in their parent scopes if they do not contain a key.

Plugins can also contain repl commands. If your plugin module has a field with a name beginning with "repl-command-" then that function will be available as a comma command from within a repl session. It will be called with a table for the repl session's environment, a function which will read the next form from stdin, a function which is used to print normal values, and one which is used to print errors.

(local fennel (require :fennel)
(fn locals [env read on-values on-error scope]
  "Print all locals in repl session scope."
  (on-values [(fennel.view env.___replLocals___)]))

{:repl-command-locals locals}

$ fennel --plugin locals-plugin.fnl
Welcome to Fennel 0.8.0 on Lua 5.4!
Use ,help to see available commands.
>> (local x 4)
nil
>> (local abc :xyz)
nil
>> ,locals
{
  :abc "xyz"
  :x 4
}

The docstring of the function will be used as its summary in the ",help" command listing. Unlike other plugin hook fields, only the first plugin to provide a repl command will be used.

Activation

Plugins are activated by passing the --plugin argument on the command line, which should be a path to a Fennel file containing a module that has some of the functions listed above. If you're using the compiler programmatically, you can include a :plugins table in the options table to most compiler entry point functions.