Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Codegen now produce models #57

Merged
merged 1 commit into from
Jun 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[run]
omit =
zero/zeromq_patterns/factory.py
zero/zeromq_patterns/helpers.py
zero/logger.py
zero/rpc/protocols.py
9 changes: 0 additions & 9 deletions Dockerfile.test.py310

This file was deleted.

9 changes: 0 additions & 9 deletions Dockerfile.test.py38

This file was deleted.

9 changes: 0 additions & 9 deletions Dockerfile.test.py39

This file was deleted.

2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ setup:
)

test:
python3 -m pytest tests --cov=zero --cov-report=term-missing -vv --durations=10 --timeout=280
python3 -m pytest tests --cov=zero --cov-report=term-missing --cov-config=.coveragerc -vv --durations=10 --timeout=280

docker-test:
docker build -t zero-test -f Dockerfile.test.py38 .
Expand Down
153 changes: 78 additions & 75 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<a href="https://codeclimate.com/github/Ananto30/zero/maintainability" target="_blank">
<img src="https://api.codeclimate.com/v1/badges/4f2fd83bee97326699bc/maintainability" />
</a>
<a href="https://pepy.tech/project/zeroapi" target="_blank">
<a href="https://pepy.tech/project/zeroapi" target="_blank">
<img src="https://static.pepy.tech/badge/zeroapi" />
</a>
</p>
Expand All @@ -27,100 +27,103 @@

**Features**:

* Zero provides **faster communication** (see [benchmarks](https://github.com/Ananto30/zero#benchmarks-)) between the microservices using [zeromq](https://zeromq.org/) under the hood.
* Zero uses messages for communication and traditional **client-server** or **request-reply** pattern is supported.
* Support for both **async** and **sync**.
* The base server (ZeroServer) **utilizes all cpu cores**.
* **Code generation**! See [example](https://github.com/Ananto30/zero#code-generation-) 👇
* Zero provides **faster communication** (see [benchmarks](https://github.com/Ananto30/zero#benchmarks-)) between the microservices using [zeromq](https://zeromq.org/) under the hood.
* Zero uses messages for communication and traditional **client-server** or **request-reply** pattern is supported.
* Support for both **async** and **sync**.
* The base server (ZeroServer) **utilizes all cpu cores**.
* **Code generation**! See [example](https://github.com/Ananto30/zero#code-generation-) 👇

**Philosophy** behind Zero:

* **Zero learning curve**: The learning curve is tends to zero. Just add functions and spin up a server, literally that's it! The framework hides the complexity of messaging pattern that enables faster communication.
* **ZeroMQ**: An awesome messaging library enables the power of Zero.
* **Zero learning curve**: The learning curve is tends to zero. Just add functions and spin up a server, literally that's it! The framework hides the complexity of messaging pattern that enables faster communication.
* **ZeroMQ**: An awesome messaging library enables the power of Zero.

Let's get started!

# Getting started 🚀

*Ensure Python 3.8+*

pip install zeroapi
```
pip install zeroapi
```

**For Windows**, [tornado](https://pypi.org/project/tornado/) needs to be installed separately (for async operations). It's not included with `zeroapi` because for linux and mac-os, tornado is not needed as they have their own event loops.

* Create a `server.py`
* Create a `server.py`

```python
from zero import ZeroServer

```python
from zero import ZeroServer
app = ZeroServer(port=5559)

app = ZeroServer(port=5559)
@app.register_rpc
def echo(msg: str) -> str:
return msg

@app.register_rpc
def echo(msg: str) -> str:
return msg
@app.register_rpc
async def hello_world() -> str:
return "hello world"

@app.register_rpc
async def hello_world() -> str:
return "hello world"

if __name__ == "__main__":
app.run()
```

if __name__ == "__main__":
app.run()
```
* The **RPC functions only support one argument** (`msg`) for now.

* The **RPC functions only support one argument** (`msg`) for now.
* Also note that server **RPC functions are type hinted**. Type hint is **must** in Zero server. Supported types can be found [here](/zero/utils/type_util.py#L11).

* Also note that server **RPC functions are type hinted**. Type hint is **must** in Zero server. Supported types can be found [here](/zero/utils/type_util.py#L11).
* Run the server

* Run the server
```shell
python -m server
```
```shell
python -m server
```

* Call the rpc methods
* Call the rpc methods

```python
from zero import ZeroClient
```python
from zero import ZeroClient

zero_client = ZeroClient("localhost", 5559)
zero_client = ZeroClient("localhost", 5559)

def echo():
resp = zero_client.call("echo", "Hi there!")
print(resp)
def echo():
resp = zero_client.call("echo", "Hi there!")
print(resp)

def hello():
resp = zero_client.call("hello_world", None)
print(resp)
def hello():
resp = zero_client.call("hello_world", None)
print(resp)


if __name__ == "__main__":
echo()
hello()
```
if __name__ == "__main__":
echo()
hello()
```

* Or using async client -
* Or using async client -

```python
import asyncio
```python
import asyncio

from zero import AsyncZeroClient
from zero import AsyncZeroClient

zero_client = AsyncZeroClient("localhost", 5559)
zero_client = AsyncZeroClient("localhost", 5559)

async def echo():
resp = await zero_client.call("echo", "Hi there!")
print(resp)
async def echo():
resp = await zero_client.call("echo", "Hi there!")
print(resp)

async def hello():
resp = await zero_client.call("hello_world", None)
print(resp)
async def hello():
resp = await zero_client.call("hello_world", None)
print(resp)


if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(echo())
loop.run_until_complete(hello())
```
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(echo())
loop.run_until_complete(hello())
```

# Serialization 📦

Expand Down Expand Up @@ -224,9 +227,9 @@ Currently, the code generation tool supports only `ZeroClient` and not `AsyncZer

# Important notes! 📝

* `ZeroServer` should always be run under `if __name__ == "__main__":`, as it uses multiprocessing.
* `ZeroServer` creates the workers in different processes, so anything global in your code will be instantiated N times where N is the number of workers. So if you want to initiate them once, put them under `if __name__ == "__main__":`. But recommended to not use global vars. And Databases, Redis, other clients, creating them N times in different processes is fine and preferred.
* The methods which are under `register_rpc()` in `ZeroServer` should have **type hinting**, like `def echo(msg: str) -> str:`
* `ZeroServer` should always be run under `if __name__ == "__main__":`, as it uses multiprocessing.
* `ZeroServer` creates the workers in different processes, so anything global in your code will be instantiated N times where N is the number of workers. So if you want to initiate them once, put them under `if __name__ == "__main__":`. But recommended to not use global vars. And Databases, Redis, other clients, creating them N times in different processes is fine and preferred.
* The methods which are under `register_rpc()` in `ZeroServer` should have **type hinting**, like `def echo(msg: str) -> str:`

# Let's do some benchmarking! 🏎

Expand All @@ -236,8 +239,8 @@ So we will be testing a gateway calling another server for some data. Check the

There are two endpoints in every tests,

* `/hello`: Just call for a hello world response 😅
* `/order`: Save a Order object in redis
* `/hello`: Just call for a hello world response 😅
* `/order`: Save a Order object in redis

Compare the results! 👇

Expand All @@ -247,23 +250,23 @@ Compare the results! 👇

*(Sorted alphabetically)*

Framework | "hello world" (req/s) | 99% latency (ms) | redis save (req/s) | 99% latency (ms)
----------- | --------------------- | ---------------- | ------------------ | ----------------
aiohttp | 14949.57 | 8.91 | 9753.87 | 13.75
aiozmq | 13844.67 | 9.55 | 5239.14 | 30.92
blacksheep | 32967.27 | 3.03 | 18010.67 | 6.79
fastApi | 13154.96 | 9.07 | 8369.87 | 15.91
sanic | 18793.08 | 5.88 | 12739.37 | 8.78
zero(sync) | 28471.47 | 4.12 | 18114.84 | 6.69
zero(async) | 29012.03 | 3.43 | 20956.48 | 5.80
| Framework | "hello world" (req/s) | 99% latency (ms) | redis save (req/s) | 99% latency (ms) |
| ----------- | --------------------- | ---------------- | ------------------ | ---------------- |
| aiohttp | 14949.57 | 8.91 | 9753.87 | 13.75 |
| aiozmq | 13844.67 | 9.55 | 5239.14 | 30.92 |
| blacksheep | 32967.27 | 3.03 | 18010.67 | 6.79 |
| fastApi | 13154.96 | 9.07 | 8369.87 | 15.91 |
| sanic | 18793.08 | 5.88 | 12739.37 | 8.78 |
| zero(sync) | 28471.47 | 4.12 | 18114.84 | 6.69 |
| zero(async) | 29012.03 | 3.43 | 20956.48 | 5.80 |

Seems like blacksheep is faster on hello world, but in more complex operations like saving to redis, zero is the winner! 🏆

# Roadmap 🗺

* [x] Make msgspec as default serializer
* [ ] Add support for async server (currently the sync server runs async functions in the eventloop, which is blocking)
* [ ] Add pub/sub support
* \[x] Make msgspec as default serializer
* \[ ] Add support for async server (currently the sync server runs async functions in the eventloop, which is blocking)
* \[ ] Add pub/sub support

# Contribution

Expand Down
5 changes: 1 addition & 4 deletions benchmarks/dockerize/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ I have used 2x cpu threads so `-t 16` and 16x25 = 400 connections.
| sanic | 13195.99 | 20.04 | 7226.72 | 25.24 |
| zero | 18867.00 | 11.48 | 12293.81 | 11.68 |


## Old benchmark results

Intel Core i3 10100, 4 cores, 8 threads, 16GB RAM, with docker limits **cpu 40% and memory 256m**
Expand All @@ -67,7 +66,6 @@ Intel Core i3 10100, 4 cores, 8 threads, 16GB RAM, with docker limits **cpu 40%
| sanic | 3,085.80 req/s | 547.02 req/s |
| zero | 5,000.77 req/s | 784.51 req/s |


MacBook Pro (13-inch, M1, 2020), Apple M1, 8 cores (4 performance and 4 efficiency), 8 GB RAM

*(Sorted alphabetically)*
Expand All @@ -81,7 +79,6 @@ MacBook Pro (13-inch, M1, 2020), Apple M1, 8 cores (4 performance and 4 efficien

More about MacBook benchmarks [here](https://github.com/Ananto30/zero/blob/main/benchmarks/others/mac-results.md)


### Note!
### Note

Please note that sometimes just `docker-compose up` will not run the `wrk`. Because you know about the docker `depends_on` only ensures the service is up, not running or healthy. So you may need to run wrk service after other services are up and running.
21 changes: 21 additions & 0 deletions examples/basic/schema.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,30 @@
from dataclasses import dataclass
from datetime import date
from typing import List

import msgspec


class Address(msgspec.Struct):
street: str
city: str
zip: int


class User(msgspec.Struct):
name: str
age: int
emails: List[str]
addresses: List[Address]
registered_at: date


@dataclass
class Teacher:
name: str


class Student(User):
roll_no: int
marks: List[int]
teachers: List[Teacher]
13 changes: 12 additions & 1 deletion examples/basic/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from zero import ZeroServer

from .schema import User
from .schema import Student, Teacher, User

app = ZeroServer(port=5559)

Expand Down Expand Up @@ -42,6 +42,17 @@ def hello_users(users: typing.List[User]) -> str:
return f"Hello {', '.join([user.name for user in users])}! Your emails are {', '.join([email for user in users for email in user.emails])}!"


teachers = [
Teacher(name="Teacher1"),
Teacher(name="Teacher2"),
]


@app.register_rpc
def hello_student(student: Student) -> str:
return f"Hello {student.name}! You are {student.age} years old. Your email is {student.emails[0]}! Your roll no. is {student.roll_no} and your marks are {student.marks}!"


if __name__ == "__main__":
app.register_rpc(echo)
app.register_rpc(hello_world)
Expand Down
Empty file.
Loading
Loading