Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(loop) separate client and loop #31

Open
wants to merge 65 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
d0ea08f
chore(*) add config files (busted/editor)
Tieske Oct 31, 2021
a4d5b8b
feat(topics) add validation and matching
Tieske Nov 3, 2021
8fff431
refactor(keepalive) remove keepalive logic from ioloop
Tieske Oct 26, 2021
6c9c48a
refactor(receive) separate out logic for handling a packet
Tieske Oct 26, 2021
d81f66c
fix(error) do not silently discard error
Tieske Oct 26, 2021
a5f5d27
refactor(locals) reduce variable scopes
Tieske Oct 27, 2021
c2956d6
feat(ping-timeout) implement ping timeout
Tieske Oct 27, 2021
af85356
feat(*) rewrite loop logic
Tieske Oct 31, 2021
136f3ce
feat(ioloop) add incremental back-off for idle clients
Tieske Nov 1, 2021
40bcd43
fix(mqtt5) pass errors on as is (like mqtt4)
Tieske Nov 4, 2021
8f8773a
wip
Tieske Nov 3, 2021
eda35db
fix(client) first_connect was not properly updated
Tieske Nov 4, 2021
567ee31
fix(tests) fix more tests, ioloop related
Tieske Nov 4, 2021
18cc55b
fix(copas) fix copas connector and test
Tieske Nov 4, 2021
13b43d4
chore(ci) add ansicolors+lualogging for testing
Tieske Nov 4, 2021
57ff134
fix(keep-alive) should always return an interval as first result
Tieske Nov 5, 2021
f6f6131
chore(examples) fix copas examples, remove sync example
Tieske Nov 5, 2021
6115148
fix(docs) update ssl defaults to match code
Tieske Nov 5, 2021
7ff684a
feat(options) allow full uri to be specified
Tieske Nov 6, 2021
30c3efc
chore(connectors) move files in subdir
Tieske Nov 5, 2021
ab736fe
chore(luasocket) integrate luasocket and luasocket-ssl
Tieske Nov 5, 2021
e34562f
feat(connector) add auto-detection for connectors
Tieske Nov 5, 2021
a600bd7
chore(test) added secure test for copas
Tieske Nov 5, 2021
fe062f6
fix(ssl) restore 'ssl_module' option and add proper checks
Tieske Nov 5, 2021
2db6e72
chore(validation) move secure conn validation+defaults to conn
Tieske Nov 7, 2021
c87bbbf
fix(reconnect) fix reconnect logic
Tieske Nov 8, 2021
587b121
feat(client) pass event handlers on init
Tieske Nov 8, 2021
ca6f39b
feat(shutdown) adds a client:shutdown method and event
Tieske Nov 8, 2021
c02e3d2
fix(rockspec) add 'connector.init' module to rockspec
Tieske Nov 8, 2021
0e15eaa
chore(examples) update to use 'opts.on' and auto-connector select
Tieske Nov 8, 2021
1fc5859
fix(openresty) fix/rewrite the openresty examples
Tieske Nov 8, 2021
dc1c261
feat(*) add nginx/copas specific code for adding clients
Tieske Nov 8, 2021
2d10112
fix(example) typos
Tieske Nov 8, 2021
b5d03b8
chore(docs) update docs
Tieske Nov 9, 2021
ca71e1b
chore(docs) reduce tab-size in docs for readability
Tieske Nov 9, 2021
1c04d08
fix(reconnect) set a number default (30 secs) if true
Tieske Nov 19, 2021
c8583cf
chore(test) rename test file
Tieske Nov 19, 2021
aa8f5c0
fix(timeouts) copas and nginx should have indefinite timeouts on reading
Tieske Nov 19, 2021
1300b0d
fix(loop) make packet handling async for copas/nginx
Tieske Nov 19, 2021
043badc
fix(send) add a lock when sedning for Copas connector
Tieske Jan 9, 2022
5bc4a92
fix(deps) bump to Copas 4
Tieske Jul 30, 2022
e9d8b39
fix(docs) minor documentation updates
Tieske Aug 7, 2022
196f7af
fix(copas) prevent accidental disabling of keep-alive
Tieske Oct 14, 2022
8bafeb6
fix(copas) switch to pause instead of sleep
Tieske Oct 26, 2022
7b8cb78
chore(docs) fix some typos
Tieske Jan 14, 2023
bbd5226
fix(patterns) escape lua magic characters
Tieske Jan 14, 2023
e452151
chore(docs) small doc updates/fixes
Tieske Jan 25, 2023
b44634a
feat(clean): option to only connect clean on first connect
Tieske Nov 24, 2023
be50bd1
change(topic): parsing topics distinguish no-match from error
Tieske Nov 25, 2023
b19fb52
chore(ci): fix linter warnings
Tieske Dec 2, 2023
b16be62
markdownlint: improved config
xHasKx Dec 22, 2023
23a070b
lua version fix md file
xHasKx Dec 22, 2023
82c6213
README.md: added short installation header
xHasKx Dec 22, 2023
aa207ac
README.md: busted tests dependency fix
xHasKx Dec 22, 2023
a84c7e4
markdown link fix
xHasKx Dec 22, 2023
c652064
markdown fix
xHasKx Dec 22, 2023
46fb48a
markdown improve
xHasKx Dec 22, 2023
fe730ba
markdown improved a bit
xHasKx Dec 22, 2023
8312df3
markdown improved a bit + openresty.lua path fix
xHasKx Dec 22, 2023
1669a15
markdown fix
xHasKx Dec 22, 2023
f8411de
markdown fix
xHasKx Dec 22, 2023
de92f3c
fix(buffered): properly calculate the remainder to be read
Tieske Jul 20, 2024
d712269
fix(luasocket): return the partial results read
Tieske Jul 20, 2024
1e2a118
docs(mqtt): fix some doc comments
Tieske Jul 20, 2024
d977890
chore(log): namespace all logs with '[LuaMQTT]'
Tieske Oct 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .busted
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
return {
default = {
ROOT = { "tests/spec" },
lpath = "./?.lua;./?/?.lua;./?/init.lua",
helper = "./tests/configure-log.lua",
verbose = true,
coverage = false,
output = "gtest",
},
}
5 changes: 5 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ root = true
[*]
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
charset = utf-8

# 4 space tab indentation for lua files
[*.lua]
Expand All @@ -20,3 +22,6 @@ indent_size = 2
[*.yml]
indent_style = space
indent_size = 2

[Makefile]
indent_style = tab
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
local/
luacov.stats.out
luacov.report.out
examples/openresty/logs/*.log
*.swp
27 changes: 14 additions & 13 deletions .luacheckrc
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,22 @@

max_line_length = 200

std = "min"

files["mqtt/**"] = {
ignore = {
"113/unpack",
"212/.+_", -- unused argument value_
}
not_globals = {
"string.len",
"table.getn",
}

files["tests/spec/**"] = {
ignore = {
"113/describe",
"113/it",
"143/assert",
}
include_files = {
"**/*.lua",
"*.rockspec",
".busted",
".luacheckrc",
}

files["tests/spec/**/*.lua"] = { std = "+busted" }
files["examples/openresty/**/*.lua"] = { std = "+ngx_lua" }
files["mqtt/loop/detect.lua"] = { std = "+ngx_lua" }
files["mqtt/loop/nginx.lua"] = { std = "+ngx_lua" }
files["mqtt/connector/nginx.lua"] = { std = "+ngx_lua" }

-- vim: ts=4 sts=4 sw=4 noet ft=lua
1 change: 1 addition & 0 deletions .markdownlint.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
default: true
single-title: false
line-length: false
code-block-style: false
4 changes: 2 additions & 2 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License
# MIT License

Copyright (c) 2018 Alexander Kiranov
Copyright (c) 2018-2021 Alexander Kiranov

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
153 changes: 29 additions & 124 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# luamqtt - Pure-lua MQTT v3.1.1 and v5.0 client
# luamqtt

Pure-lua MQTT v3.1.1 and v5.0 client

![luamqtt logo](./logo.svg)

Expand All @@ -17,12 +19,22 @@ This library is written in **pure-lua** to provide maximum portability.

* Full MQTT v3.1.1 client-side support
* Full MQTT v5.0 client-side support
* Several long-living MQTT clients in one script thanks to ioloop
* Support for Copas, OpenResty/Nginx, and an included lightweight ioloop.

# Installation

From LuaRocks:

luarocks install luamqtt

[More details](./docs_topics/01-installation.md)

# Documentation

See [https://xhaskx.github.io/luamqtt/](https://xhaskx.github.io/luamqtt/)

[More details](./docs_topics/README.md)

# Forum

See [flespi forum thread](https://forum.flespi.com/d/97-luamqtt-mqtt-client-written-in-pure-lua)
Expand All @@ -31,139 +43,32 @@ See [flespi forum thread](https://forum.flespi.com/d/97-luamqtt-mqtt-client-writ

[https://github.com/xHasKx/luamqtt](https://github.com/xHasKx/luamqtt)

# Dependencies

The only main dependency is a [**luasocket**](https://luarocks.org/modules/luasocket/luasocket) to establishing TCP connection to the MQTT broker. Install it like this:
# Bugs & contributing

```sh
luarocks install luasocket
```
Please [file a GitHub issue](https://github.com/xHasKx/luamqtt/issues) if you found any bug.

On Lua 5.1 it also depends on [**LuaBitOp**](http://bitop.luajit.org/) (**bit**) library to perform bitwise operations.
It's not listed in package dependencies, please install it manually like this:
And of course, any contribution are welcome!

```sh
luarocks install luabitop
```
# Tests

## luasec (SSL/TLS)
To run tests in this git repo you need [**busted**](https://luarocks.org/modules/olivine-labs/busted) as well as some dependencies:

To establish secure network connection (SSL/TSL) to MQTT broker
you also need [**luasec**](https://github.com/brunoos/luasec) module, please install it manually like this:
Prepare:

```sh
luarocks install busted
luarocks install luacov
luarocks install luasocket
luarocks install luasec
luarocks install copas
luarocks install lualogging
luarocks install ansicolors
```

This stage is optional and may be skipped if you don't need the secure network connection (e.g. broker is located in your local network).

# Lua versions

It's tested to work on Debian 9 GNU/Linux with Lua versions:

* Lua 5.1 ... Lua 5.4 (**i.e. any modern Lua version**)
* LuaJIT 2.0.0 ... LuaJIT 2.1.0 beta3
* It may also work on other Lua versions without any guarantees

Also I've successfully run it under **Windows** and it was ok, but installing luarock-modules may be a non-trivial task on this OS.

# Installation

As the luamqtt is almost zero-dependency you have to install required Lua libraries by yourself, before using the luamqtt library:

```sh
luarocks install luasocket # optional if you will use your own connectors (see below)
luarocks install luabitop # you don't need this for lua 5.3 and above
luarocks install luasec # you don't need this if you don't want to use SSL connections
```

Then you may install the luamqtt library itself:

```sh
luarocks install luamqtt
```

Or for development purposes;

```sh
# development branch:
luarocks install luamqtt --dev

# or from the cloned repo:
luarocks make
```

[LuaRocks page](http://luarocks.org/modules/xhaskx/luamqtt)

# Examples

Here is a short version of [`examples/simple.lua`](examples/simple.lua):

```lua
-- load mqtt library
local mqtt = require("mqtt")

-- create MQTT client, flespi tokens info: https://flespi.com/kb/tokens-access-keys-to-flespi-platform
local client = mqtt.client{ uri = "mqtt.flespi.io", username = os.getenv("FLESPI_TOKEN"), clean = true }

-- assign MQTT client event handlers
client:on{
connect = function(connack)
if connack.rc ~= 0 then
print("connection to broker failed:", connack:reason_string(), connack)
return
end

-- connection established, now subscribe to test topic and publish a message after
assert(client:subscribe{ topic="luamqtt/#", qos=1, callback=function()
assert(client:publish{ topic = "luamqtt/simpletest", payload = "hello" })
end})
end,

message = function(msg)
assert(client:acknowledge(msg))

-- receive one message and disconnect
print("received message", msg)
client:disconnect()
end,
}

-- run ioloop for client
mqtt.run_ioloop(client)
```

More examples placed in [`examples/`](examples/) directory. Also checkout tests in [`tests/spec/mqtt-client.lua`](tests/spec/mqtt-client.lua)

Also you can learn MQTT protocol by reading [`tests/spec/protocol4-make.lua`](tests/spec/protocol4-make.lua) and [`tests/spec/protocol4-parse.lua`](tests/spec/protocol4-parse.lua) tests

# Connectors

Connector is a network connection layer for luamqtt. There is a three standard connectors included:

* [`luasocket`](mqtt/luasocket.lua)
* [`luasocket_ssl`](mqtt/luasocket_ssl.lua)
* [`ngxsocket`](mqtt/ngxsocket.lua) - for using in [openresty environment](examples/openresty)

The `luasocket` or `luasocket_ssl` connector will be used by default, if not specified, according `secure=true/false` option per MQTT client.

In simple terms, connector is a set of functions to establish a network stream (TCP connection usually) and send/receive data through it.
Every MQTT client instance may have their own connector.

And it's very simple to implement your own connector to make luamqtt works in your environment.

# Bugs & contributing

Please [file a GitHub issue](https://github.com/xHasKx/luamqtt/issues) if you found any bug.

And of course, any contribution are welcome!

# Tests

To run tests in this git repo you need [**busted**](https://luarocks.org/modules/olivine-labs/busted):
Running the tests:

```sh
busted -e 'package.path="./?/init.lua;./?.lua;"..package.path' tests/spec/*.lua
busted
```

There is a script to run all tests for all supported lua versions, using [hererocks](https://github.com/mpeterv/hererocks):
Expand All @@ -180,7 +85,7 @@ To collect code coverage stats - install luacov using luarocks and then execute:

```sh
# collect stats during tests
busted -v -e 'package.path="./?/init.lua;./?.lua;"..package.path;require("luacov.runner")(".luacov")' tests/spec/*.lua
busted --coverage

# generate report into luacov.report.out file
luacov
Expand Down
28 changes: 24 additions & 4 deletions docs/config.ld
Original file line number Diff line number Diff line change
@@ -1,7 +1,21 @@
-- usage:
-- execute `ldoc .` in this docs directory

file = {"../mqtt/init.lua", "../mqtt/const.lua", "../mqtt/client.lua", "../mqtt/ioloop.lua", "../mqtt/protocol.lua"}
file = {
"../mqtt/init.lua",
"../mqtt/client.lua",
"../mqtt/ioloop.lua",

"../mqtt/loop/init.lua",
"../mqtt/loop/copas.lua",
"../mqtt/loop/nginx.lua",
"../mqtt/loop/ioloop.lua",

"../mqtt/connector/init.lua",
"../mqtt/connector/copas.lua",
"../mqtt/connector/nginx.lua",
"../mqtt/connector/luasocket.lua",
}
project = "luamqtt"
package = "mqtt"
dir = "."
Expand All @@ -12,12 +26,18 @@ full_description = "Source code: https://github.com/xHasKx/luamqtt"

examples = {
"../examples/simple.lua",
"../examples/sync.lua",
"../examples/mqtt5-simple.lua",
"../examples/copas-example.lua",
"../examples/copas.lua",
"../examples/openresty/app/openresty.lua",
}

topics = {"../README.md", "../LICENSE"}
use_markdown_titles = true
topics = {
"../README.md",
"../LICENSE",
"../docs_topics/",
}

format = "markdown"
plain = true
style = true
1 change: 1 addition & 0 deletions docs/ldoc.css
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ pre {
margin: 10px 0 10px 0;
overflow: auto;
font-family: "Andale Mono", monospace;
tab-size: 2;
}

pre.example {
Expand Down
15 changes: 15 additions & 0 deletions docs_topics/01-installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Installation

As luamqtt is almost zero-dependency you have to install any optional Lua libraries by
yourself, before using the luamqtt library.

When installing using [LuaRocks](http://luarocks.org/modules/xhaskx/luamqtt), the
LuaSocket dependency will automatically be installed as well, as it is a listed dependency
in the rockspec.

luarocks install luamqtt

To install from source clone the repo and make sure the `./mqtt/` folder is in your
Lua search path.

Check the [dependencies](./02-dependencies.md) on how (and when) to install those.
36 changes: 36 additions & 0 deletions docs_topics/02-dependencies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Dependencies

The dependencies differ slightly based on the environment you use, and the requirements you have:

* [**luasocket**](https://luarocks.org/modules/luasocket/luasocket) to establish TCP connections to the MQTT broker.
This is a listed dependency in the luamqtt rockspec, so it will automatically be installed if you use LuaRocks to
install luamqtt. To install it manually:

luarocks install luasocket

* [**copas**](https://github.com/keplerproject/copas) module for asynchoneous IO. Copas is an advanced co-routine
scheduler with far more features than the included `ioloop`. For anything more than a few devices, or for devices which
require network IO beyond mqtt alone, Copas is the better alternative. Copas is also pure-Lua, but has parallel network
IO (as opposed to sequential network IO in `ioloop`), and has features like; threads, timers, locks, semaphores, and
non-blocking clients for http(s), (s)ftp, and smtp.

luarocks install copas

* [**luasec**](https://github.com/brunoos/luasec) module for SSL/TLS based connections. This is optional and may be
skipped if you don't need secure network connections (e.g. broker is located in your local network). It's not listed
in package dependencies, please install it manually like this:

luarocks install luasec

* [**LuaBitOp**](http://bitop.luajit.org/) library to perform bitwise operations, which is required only on
Lua 5.1. It's not listed in package dependencies, please install it manually like this:

luarocks install luabitop

* [**LuaLogging**](https://github.com/lunarmodules/lualogging/) to enable logging by the MQTT client. This is optional
but highly recommended for long running clients. This is a great debugging aid when developing your clients. Also when
using OpenResty as your runtime, you'll definitely want to use this, see
[openresty.lua](https://xhaskx.github.io/luamqtt/examples/openresty.lua.html) for an example.
It's not listed in package dependencies, please install it manually like this:

luarocks install lualogging
11 changes: 11 additions & 0 deletions docs_topics/03-lua_versions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Lua versions

It's tested to work on Debian 9 GNU/Linux with Lua versions:

* Lua 5.1 ... Lua 5.4 (**i.e. any modern Lua version**)
* LuaJIT 2.0.0 ... LuaJIT 2.1.0 beta3
* It may also work on other Lua versions without any guarantees

So basically it should work on any modern Linux-based OS with Lua interpreter available.
xHasKx marked this conversation as resolved.
Show resolved Hide resolved

Also has run under **Windows** and it was ok, but installing luarocks-modules may be a non-trivial task on this OS.
Loading