finished writing tests and added documentation boilerplate

README.md

@@ -1,15 +1,25 @@
 # super-server-logs [![test](https://github.com/WiseLibs/super-server-logs/actions/workflows/test.yml/badge.svg)](https://github.com/WiseLibs/super-server-logs/actions/workflows/test.yml)
 
-This library allows you to read and write server logs. In additional to ad-hoc logging, it provides a standardized way of capturing request-response pairs and the lifecycle events of processes within a [server cluster](https://nodejs.org/api/cluster.html).
+This library allows you to read and write server logs. In additional to ad-hoc logging, it provides a standardized way of capturing *request-response pairs* and the lifecycle events of processes within a [server cluster](https://nodejs.org/api/cluster.html).
 
-* High throughput, due to batching
-* Supports time-based and size-based log rotation, with automatic file cleanup
+On the writing side, here are its notable features:
+
+* High throughput, due to batching (batch size and output latency are configurable)
+* Time-based and size-based log rotation, with automatic file cleanup (also configurable)
+* Built-in compression, resulting in smaller log files (also configurable)
 * Capable of flushing logs on process exit/crash (except on `SIGKILL` or power outage)
-* Configurable output latency and buffer size
-* Arbitrary JSON data can be included in the logs
 * Safe to use from multiple threads/processes concurrently
-* Supports log levels, where DEBUG logs are only logged when an error occurs
-* Built-in compression, resulting in smaller log files
+* DEBUG logs are only written when an error occurs
+* An efficient binary log format is used, but logs can contain arbitrary JSON data
+
+On the reading side, here are its notable features:
+
+* Quickly identify arbitrary ranges of logs based on timestamp
+	* Binary search is used under the hood, making it possible to search billions of logs in milliseconds
+* High-throughput "bulk" reading
+	* This allows you to select time ranges of logs and efficiently transfer them to external systems
+* Browser support for all read-based APIs
+* Log entries are represented as user-friendly (parsed) log objects
 
 ## Installation
 
@@ -19,11 +29,10 @@ npm install super-server-logs
 
 > Requires Node.js v18.4.x or later.
 
-## Usage
+# Documentation
 
-```js
-// TODO
-```
+- [API documentation for log **writers**](./docs/writers.md)
+- [API documentation for log **readers**](./docs/readers.md)
 
 ## License
 

docs/readers.md

@@ -0,0 +1,138 @@
+# API for reading logs
+
+- [class `LogReader`](#class-logreader)
+	- [`new LogReader(source)`](#new-logreadersource)
+	- [`reader.tail(minTimestamp[, options])`](#readertailmintimestamp-options)
+	- [`reader.range(minTimestamp, maxTimestamp)`](#readerrangemintimestamp-maxtimestamp)
+	- [`reader.rangeReversed(minTimestamp, maxTimestamp)`](#readerrangereversedmintimestamp-maxtimestamp)
+	- [`reader.bulkTail(minTimestamp[, options])`](#readerbulktailmintimestamp-options)
+	- [`reader.bulkRange(minTimestamp, maxTimestamp)`](#readerbulkrangemintimestamp-maxtimestamp)
+	- [`reader.bulkRangeReversed(minTimestamp, maxTimestamp)`](#readerbulkrangereversedmintimestamp-maxtimestamp)
+- [class `LogEntry`](#class-logentry)
+	- [`log.getRequestId()`](#loggetrequestid)
+	- [`log.getIpAddress()`](#loggetipaddress)
+	- [`log.getHttpVersion()`](#loggethttpversion)
+	- [`log.getHttpMethod()`](#loggethttpmethod)
+	- [`log.getError()`](#loggeterror)
+	- [`log.toJSON()`](#logtojson)
+- [namespace `BulkParser`](#class-bulkparser)
+	- [`BulkParser.read(chunks)`](#bulkparserreadchunks)
+	- [`BulkParser.readReversed(chunks)`](#bulkparserreadreversedchunks)
+	- [`BulkParser.parse(block)`](#bulkparserparseblock)
+- [class `LogDirectorySource`](#class-logdirectorysource)
+	- [`new LogDirectorySource(dirname[, options])`](#new-logdirectorysourcedirname-options)
+- [class `Vfs`](#class-vfs)
+	- [`new Vfs(options)`](#new-vfsoptions)
+- [enum `LogType`](#class-logtype)
+- [enum `LogLevel`](#class-loglevel)
+- [enum `Lifecycle`](#class-lifecycle)
+- [enum `HttpMethod`](#class-httpmethod)
+
+# class *LogReader*
+
+### new LogReader(source)
+
+### reader.tail(minTimestamp[, options])
+
+### reader.range(minTimestamp, maxTimestamp)
+
+### reader.rangeReversed(minTimestamp, maxTimestamp)
+
+### reader.bulkTail(minTimestamp[, options])
+
+### reader.bulkRange(minTimestamp, maxTimestamp)
+
+### reader.bulkRangeReversed(minTimestamp, maxTimestamp)
+
+# class *LogEntry*
+
+### log.getRequestId()
+
+### log.getIpAddress()
+
+### log.getHttpVersion()
+
+### log.getHttpMethod()
+
+### log.getError()
+
+### log.toJSON()
+
+# namespace *BulkParser*
+
+### BulkParser.read(chunks)
+
+### BulkParser.readReversed(chunks)
+
+### BulkParser.parse(block)
+
+# class *LogDirectorySource*
+
+### new LogDirectorySource(dirname[, options])
+
+# class *Vfs*
+
+### new Vfs(options)
+
+# enum *LogType*
+
+- `LogType.REQUEST`
+- `LogType.REQUEST_META`
+- `LogType.RESPONSE`
+- `LogType.RESPONSE_FINISHED`
+- `LogType.LOG`
+- `LogType.LIFECYCLE`
+- `LogType.UNCAUGHT_EXCEPTION`
+
+# enum *LogLevel*
+
+- `LogType.CRITICAL`
+- `LogType.ERROR`
+- `LogType.WARN`
+- `LogType.INFO`
+- `LogType.INTERNAL`
+
+# enum *Lifecycle*
+
+- `Lifecycle.WORKER_STARTED`
+- `Lifecycle.WORKER_GOING_ONLINE`
+- `Lifecycle.WORKER_ONLINE`
+- `Lifecycle.WORKER_GOING_OFFLINE`
+- `Lifecycle.WORKER_OFFLINE`
+- `Lifecycle.WORKER_DONE`
+- `Lifecycle.WORKER_PING`
+- `Lifecycle.STARTING_UP`
+- `Lifecycle.STARTING_UP_COMPLETED`
+- `Lifecycle.SHUTTING_DOWN`
+- `Lifecycle.SHUTTING_DOWN_COMPLETED`
+- `Lifecycle.WORKER_SPAWNED`
+- `Lifecycle.WORKER_EXITED`
+- `Lifecycle.MASTER_PING`
+
+# enum *HttpMethod*
+
+- `HttpMethod.GET`
+- `HttpMethod.HEAD`
+- `HttpMethod.POST`
+- `HttpMethod.PUT`
+- `HttpMethod.PATCH`
+- `HttpMethod.DELETE`
+- `HttpMethod.OPTIONS`
+- `HttpMethod.TRACE`
+- `HttpMethod.CONNECT`
+
+[any]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Data_types
+[undefined]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#undefined_type
+[null]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#null_type
+[boolean]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Boolean_type
+[number]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Number_type
+[string]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#String_type
+[Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array
+[Object]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object
+[Function]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function
+[Error]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error
+[Uint8Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array
+[Promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
+[Iterable]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterable_protocol
+[AsyncIterable]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of
+[Buffer]: https://nodejs.org/api/buffer.html#class-buffer

docs/writers.md

@@ -0,0 +1,189 @@
+# API for writing logs
+
+- [class `LogManager`](#class-logmanager)
+	- [`new LogManager(dirname[, options])`](#new-logmanagerdirname-options)
+	- [event `"rotate"`](#eventrotate)
+	- [event `"error"`](#eventerror)
+	- [`manager.close()`](#managerclose)
+	- [`manager.closed`](#managerclosed)
+	- [`manager.dirname`](#managerdirname)
+	- [`manager.filename`](#managerfilename)
+- [class `MasterLogger`](#class-masterlogger)
+	- [`new MasterLogger(filename[, options])`](#new-masterloggerfilename-options)
+	- [`masterLogger.STARTING_UP()`](#masterloggerstarting_up)
+	- [`masterLogger.STARTING_UP_COMPLETED()`](#masterloggerstarting_up_completed)
+	- [`masterLogger.SHUTTING_DOWN()`](#masterloggershutting_down)
+	- [`masterLogger.SHUTTING_DOWN_COMPLETED()`](#masterloggershutting_down_completed)
+	- [`masterLogger.WORKER_SPAWNED(workerId)`](#masterloggerworker_spawnedworkerid)
+	- [`masterLogger.WORKER_EXITED(workerId, exitCode[, signal])`](#masterloggerworker_exitedworkerid-exitcode-signal)
+	- [`masterLogger.UNCAUGHT_EXCEPTION(error)`](#masterloggeruncaught_exceptionerror)
+	- [`masterLogger.critical(data)`](#masterloggercriticaldata)
+	- [`masterLogger.error(data)`](#masterloggererrordata)
+	- [`masterLogger.warn(data)`](#masterloggerwarndata)
+	- [`masterLogger.info(data)`](#masterloggerinfodata)
+	- [`masterLogger.debug(data)`](#masterloggerdebugdata)
+	- [`masterLogger.flush()`](#masterloggerflush)
+	- [`masterLogger.rotate(filename)`](#masterloggerrotatefilename)
+	- [`masterLogger.close()`](#masterloggerclose)
+	- [`masterLogger.closed`](#masterloggerclosed)
+- [class `WorkerLogger`](#class-workerlogger)
+	- [`new WorkerLogger(filename[, options])`](#new-masterloggerfilename-options)
+	- [`workerLogger.WORKER_STARTED()`](#workerloggerworker_started)
+	- [`workerLogger.WORKER_GOING_ONLINE()`](#workerloggerworker_going_online)
+	- [`workerLogger.WORKER_ONLINE()`](#workerloggerworker_online)
+	- [`workerLogger.WORKER_GOING_OFFLINE()`](#workerloggerworker_going_offline)
+	- [`workerLogger.WORKER_OFFLINE()`](#workerloggerworker_offline)
+	- [`workerLogger.WORKER_DONE()`](#workerloggerworker_done)
+	- [`workerLogger.UNCAUGHT_EXCEPTION(error)`](#workerloggeruncaught_exceptionerror)
+	- [`workerLogger.newRequest()`](#workerloggernewrequest)
+	- [`workerLogger.critical(data)`](#workerloggercriticaldata)
+	- [`workerLogger.error(data)`](#workerloggererrordata)
+	- [`workerLogger.warn(data)`](#workerloggerwarndata)
+	- [`workerLogger.info(data)`](#workerloggerinfodata)
+	- [`workerLogger.debug(data)`](#workerloggerdebugdata)
+	- [`workerLogger.flush()`](#workerloggerflush)
+	- [`workerLogger.rotate(filename)`](#workerloggerrotatefilename)
+	- [`workerLogger.close()`](#workerloggerclose)
+	- [`workerLogger.closed`](#workerloggerclosed)
+- [class `RequestLogger`](#class-requestlogger)
+	- [`requestLogger.REQUEST(req)`](#requestloggerrequestreq)
+	- [`requestLogger.REQUEST_META(data)`](#requestloggerrequest_metadata)
+	- [`requestLogger.RESPONSE(statusCode[, error])`](#requestloggerresponsestatuscode-error)
+	- [`requestLogger.RESPONSE_FINISHED([error])`](#requestloggerresponse_finishederror)
+	- [`requestLogger.critical(data)`](#requestloggercriticaldata)
+	- [`requestLogger.error(data)`](#requestloggererrordata)
+	- [`requestLogger.warn(data)`](#requestloggerwarndata)
+	- [`requestLogger.info(data)`](#requestloggerinfodata)
+	- [`requestLogger.debug(data)`](#requestloggerdebugdata)
+	- [`requestLogger.closed`](#requestloggerclosed)
+	- [`requestLogger.requestId`](#requestloggerrequestid)
+
+# class *LogManager*
+
+### new LogManager(dirname[, options])
+
+### event `"rotate"`
+
+### event `"error"`
+
+### manager.close()
+
+### manager.closed
+
+### manager.dirname
+
+### manager.filename
+
+# class *MasterLogger*
+
+### new MasterLogger(filename[, options])
+
+### masterLogger.STARTING_UP()
+
+### masterLogger.STARTING_UP_COMPLETED()
+
+### masterLogger.SHUTTING_DOWN()
+
+### masterLogger.SHUTTING_DOWN_COMPLETED()
+
+### masterLogger.WORKER_SPAWNED(workerId)
+
+### masterLogger.WORKER_EXITED(workerId, exitCode[, signal])
+
+### masterLogger.UNCAUGHT_EXCEPTION(error)
+
+### masterLogger.critical(data)
+
+### masterLogger.error(data)
+
+### masterLogger.warn(data)
+
+### masterLogger.info(data)
+
+### masterLogger.debug(data)
+
+### masterLogger.flush()
+
+### masterLogger.rotate(filename)
+
+### masterLogger.close()
+
+### masterLogger.closed
+
+# class *WorkerLogger*
+
+### new WorkerLogger(filename[, options])
+
+### workerLogger.WORKER_STARTED()
+
+### workerLogger.WORKER_GOING_ONLINE()
+
+### workerLogger.WORKER_ONLINE()
+
+### workerLogger.WORKER_GOING_OFFLINE()
+
+### workerLogger.WORKER_OFFLINE()
+
+### workerLogger.WORKER_DONE()
+
+### workerLogger.UNCAUGHT_EXCEPTION(error)
+
+### workerLogger.newRequest()
+
+### workerLogger.critical(data)
+
+### workerLogger.error(data)
+
+### workerLogger.warn(data)
+
+### workerLogger.info(data)
+
+### workerLogger.debug(data)
+
+### workerLogger.flush()
+
+### workerLogger.rotate(filename)
+
+### workerLogger.close()
+
+### workerLogger.closed
+
+# class *RequestLogger*
+
+### requestLogger.REQUEST(req)
+
+### requestLogger.REQUEST_META(data)
+
+### requestLogger.RESPONSE(statusCode[, error])
+
+### requestLogger.RESPONSE_FINISHED([error])
+
+### requestLogger.critical(data)
+
+### requestLogger.error(data)
+
+### requestLogger.warn(data)
+
+### requestLogger.info(data)
+
+### requestLogger.debug(data)
+
+### requestLogger.closed
+
+### requestLogger.requestId
+
+[any]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Data_types
+[undefined]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#undefined_type
+[null]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#null_type
+[boolean]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Boolean_type
+[number]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Number_type
+[string]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#String_type
+[Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array
+[Object]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object
+[Function]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function
+[Error]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error
+[Uint8Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Uint8Array
+[Promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
+[Iterable]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols#the_iterable_protocol
+[AsyncIterable]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of
+[Buffer]: https://nodejs.org/api/buffer.html#class-buffer

src/nodejs/log-directory-source.js

@@ -228,6 +228,9 @@ async function isRotating(file) {
 				return false;
 			} else if (!pendingWorkers) {
 				if (log.event === MASTER_PING) {
+					if (!log.workerIds.length) {
+						return false;
+					}
 					pendingWorkers = new Set(log.workerIds);
 				}
 			} else if (log.workerId != null) {