Skip to content

Commit

Permalink
Codegen now produce models
Browse files Browse the repository at this point in the history
  • Loading branch information
Ananto30 committed Jun 28, 2024
1 parent c05b03a commit fc49994
Show file tree
Hide file tree
Showing 31 changed files with 2,080 additions and 353 deletions.
6 changes: 6 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[run]
omit =
zero/zeromq_patterns/factory.py
zero/zeromq_patterns/helpers.py
zero/logger.py
zero/rpc/protocols.py
9 changes: 0 additions & 9 deletions Dockerfile.test.py310

This file was deleted.

9 changes: 0 additions & 9 deletions Dockerfile.test.py38

This file was deleted.

9 changes: 0 additions & 9 deletions Dockerfile.test.py39

This file was deleted.

2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ setup:
)

test:
python3 -m pytest tests --cov=zero --cov-report=term-missing -vv --durations=10 --timeout=280
python3 -m pytest tests --cov=zero --cov-report=term-missing --cov-config=.coveragerc -vv --durations=10 --timeout=280

docker-test:
docker build -t zero-test -f Dockerfile.test.py38 .
Expand Down
153 changes: 78 additions & 75 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<a href="https://codeclimate.com/github/Ananto30/zero/maintainability" target="_blank">
<img src="https://api.codeclimate.com/v1/badges/4f2fd83bee97326699bc/maintainability" />
</a>
<a href="https://pepy.tech/project/zeroapi" target="_blank">
<a href="https://pepy.tech/project/zeroapi" target="_blank">
<img src="https://static.pepy.tech/badge/zeroapi" />
</a>
</p>
Expand All @@ -27,100 +27,103 @@

**Features**:

* Zero provides **faster communication** (see [benchmarks](https://github.com/Ananto30/zero#benchmarks-)) between the microservices using [zeromq](https://zeromq.org/) under the hood.
* Zero uses messages for communication and traditional **client-server** or **request-reply** pattern is supported.
* Support for both **async** and **sync**.
* The base server (ZeroServer) **utilizes all cpu cores**.
* **Code generation**! See [example](https://github.com/Ananto30/zero#code-generation-) 👇
* Zero provides **faster communication** (see [benchmarks](https://github.com/Ananto30/zero#benchmarks-)) between the microservices using [zeromq](https://zeromq.org/) under the hood.
* Zero uses messages for communication and traditional **client-server** or **request-reply** pattern is supported.
* Support for both **async** and **sync**.
* The base server (ZeroServer) **utilizes all cpu cores**.
* **Code generation**! See [example](https://github.com/Ananto30/zero#code-generation-) 👇

**Philosophy** behind Zero:

* **Zero learning curve**: The learning curve is tends to zero. Just add functions and spin up a server, literally that's it! The framework hides the complexity of messaging pattern that enables faster communication.
* **ZeroMQ**: An awesome messaging library enables the power of Zero.
* **Zero learning curve**: The learning curve is tends to zero. Just add functions and spin up a server, literally that's it! The framework hides the complexity of messaging pattern that enables faster communication.
* **ZeroMQ**: An awesome messaging library enables the power of Zero.

Let's get started!

# Getting started 🚀

*Ensure Python 3.8+*

pip install zeroapi
```
pip install zeroapi
```

**For Windows**, [tornado](https://pypi.org/project/tornado/) needs to be installed separately (for async operations). It's not included with `zeroapi` because for linux and mac-os, tornado is not needed as they have their own event loops.

* Create a `server.py`
* Create a `server.py`

```python
from zero import ZeroServer

```python
from zero import ZeroServer
app = ZeroServer(port=5559)

app = ZeroServer(port=5559)
@app.register_rpc
def echo(msg: str) -> str:
return msg

@app.register_rpc
def echo(msg: str) -> str:
return msg
@app.register_rpc
async def hello_world() -> str:
return "hello world"

@app.register_rpc
async def hello_world() -> str:
return "hello world"

if __name__ == "__main__":
app.run()
```

if __name__ == "__main__":
app.run()
```
* The **RPC functions only support one argument** (`msg`) for now.

* The **RPC functions only support one argument** (`msg`) for now.
* Also note that server **RPC functions are type hinted**. Type hint is **must** in Zero server. Supported types can be found [here](/zero/utils/type_util.py#L11).

* Also note that server **RPC functions are type hinted**. Type hint is **must** in Zero server. Supported types can be found [here](/zero/utils/type_util.py#L11).
* Run the server

* Run the server
```shell
python -m server
```
```shell
python -m server
```

* Call the rpc methods
* Call the rpc methods

```python
from zero import ZeroClient
```python
from zero import ZeroClient

zero_client = ZeroClient("localhost", 5559)
zero_client = ZeroClient("localhost", 5559)

def echo():
resp = zero_client.call("echo", "Hi there!")
print(resp)
def echo():
resp = zero_client.call("echo", "Hi there!")
print(resp)

def hello():
resp = zero_client.call("hello_world", None)
print(resp)
def hello():
resp = zero_client.call("hello_world", None)
print(resp)


if __name__ == "__main__":
echo()
hello()
```
if __name__ == "__main__":
echo()
hello()
```

* Or using async client -
* Or using async client -

```python
import asyncio
```python
import asyncio

from zero import AsyncZeroClient
from zero import AsyncZeroClient

zero_client = AsyncZeroClient("localhost", 5559)
zero_client = AsyncZeroClient("localhost", 5559)

async def echo():
resp = await zero_client.call("echo", "Hi there!")
print(resp)
async def echo():
resp = await zero_client.call("echo", "Hi there!")
print(resp)

async def hello():
resp = await zero_client.call("hello_world", None)
print(resp)
async def hello():
resp = await zero_client.call("hello_world", None)
print(resp)


if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(echo())
loop.run_until_complete(hello())
```
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(echo())
loop.run_until_complete(hello())
```

# Serialization 📦

Expand Down Expand Up @@ -224,9 +227,9 @@ Currently, the code generation tool supports only `ZeroClient` and not `AsyncZer

# Important notes! 📝

* `ZeroServer` should always be run under `if __name__ == "__main__":`, as it uses multiprocessing.
* `ZeroServer` creates the workers in different processes, so anything global in your code will be instantiated N times where N is the number of workers. So if you want to initiate them once, put them under `if __name__ == "__main__":`. But recommended to not use global vars. And Databases, Redis, other clients, creating them N times in different processes is fine and preferred.
* The methods which are under `register_rpc()` in `ZeroServer` should have **type hinting**, like `def echo(msg: str) -> str:`
* `ZeroServer` should always be run under `if __name__ == "__main__":`, as it uses multiprocessing.
* `ZeroServer` creates the workers in different processes, so anything global in your code will be instantiated N times where N is the number of workers. So if you want to initiate them once, put them under `if __name__ == "__main__":`. But recommended to not use global vars. And Databases, Redis, other clients, creating them N times in different processes is fine and preferred.
* The methods which are under `register_rpc()` in `ZeroServer` should have **type hinting**, like `def echo(msg: str) -> str:`

# Let's do some benchmarking! 🏎

Expand All @@ -236,8 +239,8 @@ So we will be testing a gateway calling another server for some data. Check the

There are two endpoints in every tests,

* `/hello`: Just call for a hello world response 😅
* `/order`: Save a Order object in redis
* `/hello`: Just call for a hello world response 😅
* `/order`: Save a Order object in redis

Compare the results! 👇

Expand All @@ -247,23 +250,23 @@ Compare the results! 👇

*(Sorted alphabetically)*

Framework | "hello world" (req/s) | 99% latency (ms) | redis save (req/s) | 99% latency (ms)
----------- | --------------------- | ---------------- | ------------------ | ----------------
aiohttp | 14949.57 | 8.91 | 9753.87 | 13.75
aiozmq | 13844.67 | 9.55 | 5239.14 | 30.92
blacksheep | 32967.27 | 3.03 | 18010.67 | 6.79
fastApi | 13154.96 | 9.07 | 8369.87 | 15.91
sanic | 18793.08 | 5.88 | 12739.37 | 8.78
zero(sync) | 28471.47 | 4.12 | 18114.84 | 6.69
zero(async) | 29012.03 | 3.43 | 20956.48 | 5.80
| Framework | "hello world" (req/s) | 99% latency (ms) | redis save (req/s) | 99% latency (ms) |
| ----------- | --------------------- | ---------------- | ------------------ | ---------------- |
| aiohttp | 14949.57 | 8.91 | 9753.87 | 13.75 |
| aiozmq | 13844.67 | 9.55 | 5239.14 | 30.92 |
| blacksheep | 32967.27 | 3.03 | 18010.67 | 6.79 |
| fastApi | 13154.96 | 9.07 | 8369.87 | 15.91 |
| sanic | 18793.08 | 5.88 | 12739.37 | 8.78 |
| zero(sync) | 28471.47 | 4.12 | 18114.84 | 6.69 |
| zero(async) | 29012.03 | 3.43 | 20956.48 | 5.80 |

Seems like blacksheep is faster on hello world, but in more complex operations like saving to redis, zero is the winner! 🏆

# Roadmap 🗺

* [x] Make msgspec as default serializer
* [ ] Add support for async server (currently the sync server runs async functions in the eventloop, which is blocking)
* [ ] Add pub/sub support
* \[x] Make msgspec as default serializer
* \[ ] Add support for async server (currently the sync server runs async functions in the eventloop, which is blocking)
* \[ ] Add pub/sub support

# Contribution

Expand Down
5 changes: 1 addition & 4 deletions benchmarks/dockerize/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ I have used 2x cpu threads so `-t 16` and 16x25 = 400 connections.
| sanic | 13195.99 | 20.04 | 7226.72 | 25.24 |
| zero | 18867.00 | 11.48 | 12293.81 | 11.68 |


## Old benchmark results

Intel Core i3 10100, 4 cores, 8 threads, 16GB RAM, with docker limits **cpu 40% and memory 256m**
Expand All @@ -67,7 +66,6 @@ Intel Core i3 10100, 4 cores, 8 threads, 16GB RAM, with docker limits **cpu 40%
| sanic | 3,085.80 req/s | 547.02 req/s |
| zero | 5,000.77 req/s | 784.51 req/s |


MacBook Pro (13-inch, M1, 2020), Apple M1, 8 cores (4 performance and 4 efficiency), 8 GB RAM

*(Sorted alphabetically)*
Expand All @@ -81,7 +79,6 @@ MacBook Pro (13-inch, M1, 2020), Apple M1, 8 cores (4 performance and 4 efficien

More about MacBook benchmarks [here](https://github.com/Ananto30/zero/blob/main/benchmarks/others/mac-results.md)


### Note!
### Note

Please note that sometimes just `docker-compose up` will not run the `wrk`. Because you know about the docker `depends_on` only ensures the service is up, not running or healthy. So you may need to run wrk service after other services are up and running.
21 changes: 21 additions & 0 deletions examples/basic/schema.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,30 @@
from dataclasses import dataclass
from datetime import date
from typing import List

import msgspec


class Address(msgspec.Struct):
street: str
city: str
zip: int


class User(msgspec.Struct):
name: str
age: int
emails: List[str]
addresses: List[Address]
registered_at: date


@dataclass
class Teacher:
name: str


class Student(User):
roll_no: int
marks: List[int]
teachers: List[Teacher]
13 changes: 12 additions & 1 deletion examples/basic/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from zero import ZeroServer

from .schema import User
from .schema import Student, Teacher, User

app = ZeroServer(port=5559)

Expand Down Expand Up @@ -42,6 +42,17 @@ def hello_users(users: typing.List[User]) -> str:
return f"Hello {', '.join([user.name for user in users])}! Your emails are {', '.join([email for user in users for email in user.emails])}!"


teachers = [
Teacher(name="Teacher1"),
Teacher(name="Teacher2"),
]


@app.register_rpc
def hello_student(student: Student) -> str:
return f"Hello {student.name}! You are {student.age} years old. Your email is {student.emails[0]}! Your roll no. is {student.roll_no} and your marks are {student.marks}!"


if __name__ == "__main__":
app.register_rpc(echo)
app.register_rpc(hello_world)
Expand Down
Empty file.
Loading

0 comments on commit fc49994

Please sign in to comment.