Skip to content

Commit

Permalink
release 0.1.0
Browse files Browse the repository at this point in the history
  • Loading branch information
berislavlopac committed Dec 13, 2022
1 parent 7018a2e commit 173134e
Show file tree
Hide file tree
Showing 12 changed files with 214 additions and 90 deletions.
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Unclogger
=========

Simple library for customisable structured logging.]
Simple library for customisable structured logging.


## Development Environment
Expand Down Expand Up @@ -54,5 +54,3 @@ $ mkdocs build
```

This will create the HTML documentation in the `site` directory.

API Documentation is served by GitLab Pages.
Binary file added docs/assets/plunger-favicon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
97 changes: 55 additions & 42 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,19 @@

`unclogger` is a simple library for customisable structured logging. It mirrors the standard Python logging library API, with a few additional functionalities.

## Structured Logging
!!! Info

Implementation detail: `unclogger` is using the [Structlog library](https://www.structlog.org) under the hood.

## Structured Loggers

Passing keyword arguments to a log method will add them to the log data. Log data is converted to a JSON message before sending.

**Note:** The JSON in examples below was formatted for convenience; in practice it is sent as a single line of text.

!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger")
>>> logger.info("test test", foo="abc", bar=123)
Expand All @@ -19,12 +26,16 @@ Passing keyword arguments to a log method will add them to the log data. Log dat
"level": "info",
"timestamp": "2021-02-12T22:40:07.600385Z"
}

>>>
```

### Configuration

The logger can be configured using the special attribute `config`:
The logger can be configured using the special attribute `config`. This can be used for configuring additional functionality, e.g. when [handling sensitive data](sensitive.md).

!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger")
>>> logger.config.foo = "foo"
Expand All @@ -37,41 +48,19 @@ The logger can be configured using the special attribute `config`:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'types.SimpleNamespace' object has no attribute 'baz'. Did you mean: 'bar'?
>>>
```

As can be seen in the error message above, `config` is an instance of [SimpleNamespace](https://docs.python.org/3.10/library/types.html#types.SimpleNamespace).


### Global Context Binding

The [`context_bind`](reference.md#uncloggerloggercontext_bind) function will set values in the global context, where it can be used by any logger.
### Local Context

>>> from unclogger import get_logger, bind_data
>>> # binding data before even creating a logger
>>> bind_data(abc="def")
>>> logger1 = get_logger("test logger 1")
>>> logger1.info("test test", foo="abc", bar=123)
{
"abc": "def",
"foo": "abc",
"bar": 123,
"event": "test test",
"logger": "test logger 1",
"level": "info",
"timestamp": "2021-02-12T22:43:48.062282Z"
}
>>> logger2 = get_logger("test logger 2")
>>> # a different logger can access the same data
>>> logger2.info("another message")
{
"abc": "def",
"event": "another message",
"logger": "test logger 2",
"level": "info", "timestamp":
"2021-02-12T22:45:05.599852Z"
}
Each logger has a local context; values can be bound to it so they can appear in any message sent by that logger.

### Local Context Binding
!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger").bind(abc="def")
>>> logger.info("test test", foo="abc", bar=123)
Expand Down Expand Up @@ -110,16 +99,40 @@ The [`context_bind`](reference.md#uncloggerloggercontext_bind) function will set
"level": "info",
"timestamp": "2021-02-12T23:08:21.768578Z"
}
>>>
```

### Logging Level Configuration

>>> from unclogger import get_logger, configure
>>> logger = get_logger("test logger")
>>> logger.info("bar")
{"event": "bar", "logger": "test logger", "level": "info", "timestamp": "2021-02-18T21:59:40.102272Z"}
>>> logger.debug("bar")
>>> configure("debug")
>>> logger.debug("bar")
{"event": "bar", "logger": "test logger", "level": "debug", "timestamp": "2021-02-18T22:00:09.147106Z"}
>>> configure("warning")
>>> logger.info("bar")
## Global Context

The [`context_bind`](reference.md#uncloggerloggercontext_bind) function will set values in the global context, where it can be used by any logger.

!!! Example

```python
>>> from unclogger import get_logger, context_bind
>>> # binding data before even creating a logger
>>> context_bind(abc="def")
>>> logger1 = get_logger("test logger 1")
>>> logger1.info("test test", foo="abc", bar=123)
{
"abc": "def",
"foo": "abc",
"bar": 123,
"event": "test test",
"logger": "test logger 1",
"level": "info",
"timestamp": "2021-02-12T22:43:48.062282Z"
}
>>> logger2 = get_logger("test logger 2")
>>> # a different logger can access the same data
>>> logger2.info("another message")
{
"abc": "def",
"event": "another message",
"logger": "test logger 2",
"level": "info", "timestamp":
"2021-02-12T22:45:05.599852Z"
}
>>>
```
42 changes: 39 additions & 3 deletions docs/reference.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,43 @@
# API Reference

## ![mkapi](unclogger.get_logger)
## Logger

## ![mkapi](unclogger.context_bind)
### ![mkapi](unclogger.get_logger)

## ![mkapi](unclogger.configure)
### ![mkapi](unclogger.context_bind)

## Global Log Level Configuration

??? Example

```python
>>> from unclogger import get_logger, set_level
>>> logger = get_logger("test logger")
>>> logger.info("bar")
{
"event": "bar",
"logger": "test logger",
"level": "info",
"timestamp": "2021-02-18T21:59:40.102272Z"
}
>>> logger.debug("bar")
>>> set_level("debug")
>>> logger.debug("bar")
{
"event": "bar",
"logger": "test logger",
"level": "debug",
"timestamp": "2021-02-18T22:00:09.147106Z"
}
>>> set_level("warning")
>>> logger.info("bar")
>>>
```

### ![mkapi](unclogger.set_level)

## Processors

### Sensitive Data

#### ![mkapi](unclogger.processors.clean_data.clean_sensitive_data)
31 changes: 31 additions & 0 deletions docs/sensitive.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

If the name of any field in the structured log message matches one of the listed sensitive names, the value of that field is (recursively) replaced with a safe value:

!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger")
>>> logger.info("clean password", password="blabla", foo={"Email": "[email protected]"})
Expand All @@ -18,19 +21,29 @@ If the name of any field in the structured log message matches one of the listed
"level": "info",
"timestamp": "2022-02-02T10:53:52.245833Z"
}
>>>
```

A basic list of sensitive field names is included in `unclogger`:

!!! Example

```python
>>> from unclogger.processors.clean_data import SENSITIVE_FIELDS
>>> SENSITIVE_FIELDS
['password', 'email', 'email_1', 'firstname', 'lastname', 'currentpassword', 'newpassword', 'tmppassword', 'authentication', 'refresh', 'auth', 'http_refresh', 'http_x_forwarded_authorization', 'http_x_endpoint_api_userinfo', 'http_authorization', 'idtoken', 'oauthidtoken', 'publickey', 'privatekey']
>>>
```

!!! Note

Note that the list is case-insensitive; `unclogger` normalizes all field names to lowercase, so e.g. `email` and `Email` are treated equally.

This list can be configured with an iterable of custom field names:

!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger")
>>> logger.config.sensitive_keys = {"foo", "bar"}
Expand All @@ -49,11 +62,16 @@ This list can be configured with an iterable of custom field names:
"timestamp":
"2022-02-02T11:08:01.260019Z"
}
>>>
```

### Configurable Replacement Value

A custom string can be used instead of the default replacement value:

!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger")
>>> logger.config.replacement = "blablabla"
Expand All @@ -66,11 +84,16 @@ A custom string can be used instead of the default replacement value:
"level": "info",
"timestamp": "2022-12-13T20:02:38.520599Z"
}
>>>
```

### Hashing Sensitive Data

Instead of a replacement string, `config.replacement` can define a Python callable:

!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger")
>>> logger.config.replacement = hashlib.sha256
Expand All @@ -83,6 +106,8 @@ Instead of a replacement string, `config.replacement` can define a Python callab
"level": "info",
"timestamp": "2022-12-13T20:06:37.542212Z"
}
>>>
```

This can be used so that the data can still be identified (e.g. an email address will always have the same has value) without sending the actual data to the log.

Expand All @@ -98,6 +123,10 @@ This can be used so that the data can still be identified (e.g. an email address
```

## Sensitive Text Values

!!! Example

```python
>>> from unclogger import get_logger
>>> logger = get_logger("test logger")
>>> logger.info("'Authentication': 1234")
Expand All @@ -107,5 +136,7 @@ This can be used so that the data can still be identified (e.g. an email address
"level": "info",
"timestamp": "2022-02-02T11:22:21.997204Z"
}
>>>
```

*[PII]: Personally Identifiable Information
6 changes: 4 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,17 +1,19 @@
site_name: Unclogger
repo_url: https://github/berislavlopac/unclogger
repo_url: https://github.com/berislavlopac/unclogger
site_description: Simple library for customisable structured logging.]
site_author: Berislav Lopac <[email protected]>
use_directory_urls: false
theme:
name: material
logo: assets/plunger.png
logo: assets/plunger-logo.png
favicon: assets/plunger-favicon.png
plugins:
- search
- mkapi
markdown_extensions:
- abbr
- attr_list
- pymdownx.details
- pymdownx.highlight:
anchor_linenums: true
- pymdownx.inlinehilite
Expand Down
8 changes: 0 additions & 8 deletions setup.cfg

This file was deleted.

22 changes: 11 additions & 11 deletions tests/logger/test_configure.py → tests/logger/test_set_level.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,35 +2,35 @@

import pytest

from unclogger import configure
from unclogger import set_level


def test_configure_sets_default_log_level():
def test_sets_default_log_level():
# configure() is called at import time
assert logging.root.level == logging.INFO


def test_configure_sets_log_level_with_textual_parameter(caplog):
def test_sets_log_level_with_textual_parameter(caplog):
with caplog.at_level(logging.INFO):
assert logging.root.level == logging.INFO
configure("DEBUG")
set_level("DEBUG")
assert logging.root.level == logging.DEBUG
configure("error")
set_level("error")
assert logging.root.level == logging.ERROR


def test_configure_sets_log_level_with_integer_parameter(caplog):
def test_sets_log_level_with_integer_parameter(caplog):
with caplog.at_level(logging.INFO):
assert logging.root.level == logging.INFO
configure(logging.DEBUG)
set_level(logging.DEBUG)
assert logging.root.level == logging.DEBUG
configure(40)
set_level(40)
assert logging.root.level == logging.ERROR
configure("30")
set_level("30")
assert logging.root.level == logging.WARNING


@pytest.mark.parametrize("value", ("foo", "bar baz", 12.5, "12.5"))
def test_configure_raises_an_exception_on_incorrect_argument(value):
def test_raises_an_exception_on_incorrect_argument(value):
with pytest.raises(ValueError):
configure(value)
set_level(value)
7 changes: 5 additions & 2 deletions unclogger/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
"""Simple library for customisable structured logging."""

from .logger import configure, context_bind, context_clear, get_logger, Unclogger
import logging as _std_logging

from .logger import context_bind, context_clear, get_logger, set_level, Unclogger

getLogger = get_logger # alias for compatibility with standard logging

configure()
_std_logging.basicConfig(format="%(message)s")
set_level()
Loading

0 comments on commit 173134e

Please sign in to comment.