Home

Welcome to the moler wiki!

Moler name origin

Name is coined by Grzegorz Latuszek with high impact of Bartosz Odziomek, Michał Ernst and Mateusz Smet.

Moler comes from:

• Mole 🇬🇧
  • has tunnel-connections to search for data (:bug:) it is processing
  • can detect different bugs hidden under ground
  • as we want this library to search/detect bugs in tested software
• Moler in spanish 🇪🇸 means:
  • grind, reduce to powder
  • as this library should grind tested software to find it's bugs

Architecture - fundamental idea

Architectural idea behind this library is separation of 3 abstractions:

1. commands/connection-observers
2. concurrency
3. I/O connections

This is to allow reusing the first one (main focus of Moler) with any concurrency/IO. That also means independent development of the 3 abstractions.

What I mean concurrency? Anything that allows given command to "run in background" and then "await till it is done". It may be realized by threads, twisted, asyncio, curio, ... .

But, at the end I want simplicity for end user. She/he should be able to just say "run in background" without knowing "how".

Architecture inspirations

As it comes to I/O I've been insiped by “bring-your-own-I/O” idea of hyper-h2 and h11. This idea minimizes dependency on I/O, allows code reuse under different I/O.

As it comes to layering concept I've been inspired by Cory Benfield's PYCON UK 2015: Simplicity Is A Feature

However, “bring-your-own-I/O” applies only to Layer-1. Next layers introduce some implementation of I/O provided by Moler but you can also switch them to your own. That way user gains next-level-of-simplicity "no need to write I/O, take what is given".

Architecture layers

We will go over Moler's architecture bottom-up.

Lowest layer gives you control over all details. You may combine all pieces as you wish but the price for it is more effort on your side, more code to glue them all. Up in layers means less code to write for the price of less freedom.

Layer 1 - glue it all manually (under your concurrency-I/O)

• External-IO-connection just passes bytes
• Moler's connection perform encoding/decoding and dispatching to multiple observers doing "parallel" observation

Mole icon made from Icon Fonts is licensed by CC BY 3.0

Layer 2 - just use it

This layer introduces concept of runners - entities that are used internally by connection observers to realize "background run". Allows to write code like (see layer_2 examples):

# using connection-observer as future:
net_down_detector.start()
# ...
net_down_time = net_down_detector.await_done(timeout=10)

# or using it as synchronous function
detect_network_up = net_up_detector  # function, so we want verb to express action
net_up_time = detect_network_up()

Concurrency here is used for two purposes:

1. background run of commands/connection-observers
2. self-running connections - after they are open they can read data as it comes (not delayed by lack of control) External I/O variants are:

• variants per I/O type: tcp, udp, ssh, process, shell, ...
• multiplied by varians of concurrency: tcp on asyncio, tcp on twisted, socket tcp in thread, ...

Layer 3 - configurable choice

User will select concurrency/IO via configuration.

Choice is done on backend via configuration. This alows to write code like

my_remote_machine = get_connection(type="tcp", host="hostname", port=87995)

and thats it (frontend usage)!!! Why should user bother if it is asyncio-TCP or Twisted-TCP?

Configuration may be done by python code or read from config files.

Layer 4 - autodetected variants

Modules for selecting configuration (concurrency/IO variants) based on software available on given machine.

Like: "asyncio only if you have Python3".