Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor the ratelimiting behavior out of the request handlers. #226

Merged
merged 4 commits into from
Mar 11, 2024

Conversation

shtlrs
Copy link
Contributor

@shtlrs shtlrs commented Jan 8, 2024

As I was browsing the code, I noticed that all get and post handlers had to check for ratelimits before handling the request.

This had been ported over to a ratelimit decorator that we use to wrap any http verb's callback function with the ratelimiting behavior.

I initially implemented this as a Mixin, but wookie suggested that a per-handler decorator would be better, which makes sense, so it has been changed to that.

Thanks @wookie184 for the alternative approach idea :)

@shtlrs shtlrs force-pushed the ratelimit-mixin branch 2 times, most recently from e0daa7b to 41f966d Compare January 8, 2024 14:45
@supakeen
Copy link
Owner

supakeen commented Jan 9, 2024

This one needs a bit of review; I'll read it when I have some spare time :)

@supakeen supakeen self-assigned this Jan 9, 2024
Copy link
Contributor

@wookie184 wookie184 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO a cleaner approach could be a per method decorator, as I think it makes more sense for ratelimits to be per method. Then you would have something like this:

class Lexer(Base):
    @ratelimit_endpoint(area="read")
    async def get(self) -> None:
        self.write(utility.list_languages())

I also think that if the change is made here it should be applied to website.py for consistency. Not sure about api_curl or api_deprecated, they could probably be left as is.

src/pinnwand/handler/api_v1.py Show resolved Hide resolved
@shtlrs
Copy link
Contributor Author

shtlrs commented Jan 19, 2024

@wookie184 Good point, I'll make it a decorator instead of a mixin.

@shtlrs shtlrs force-pushed the ratelimit-mixin branch 2 times, most recently from 9abf110 to 9005e19 Compare January 19, 2024 23:28
@shtlrs
Copy link
Contributor Author

shtlrs commented Jan 19, 2024

IMO a cleaner approach could be a per method decorator, as I think it makes more sense for ratelimits to be per method. Then you would have something like this:

class Lexer(Base):
    @ratelimit_endpoint(area="read")
    async def get(self) -> None:
        self.write(utility.list_languages())

Redid the commits to use a decorator instead of a mixin.

I also think that if the change is made here it should be applied to website.py for consistency. Not sure about api_curl or api_deprecated, they could probably be left as is.

I ended up applying it all over.

@shtlrs shtlrs force-pushed the ratelimit-mixin branch 2 times, most recently from 63c1bfb to a16af3b Compare January 20, 2024 00:17
Comment on lines 61 to 75
@wraps(func)
def inner(*args, **kwargs):
# We transform the args into a dict because ChainMap works differently starting python 3.9
# In previous versions, it expects all maps to be dictionaries, whereas starting py3.9
# It transforms list/tuples into a dict whose keys are the different elements.
arguments = ChainMap(dict.fromkeys(args), kwargs.values())
request_handler = None
for arg in arguments:
if isinstance(arg, RequestHandler):
request_handler = arg
break

if should_be_ratelimited(request_handler.request, area):
raise error.RatelimitError()
return func(*args, **kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's guaranteed that the first argument will always be self which is the request handler, does this work?

Suggested change
@wraps(func)
def inner(*args, **kwargs):
# We transform the args into a dict because ChainMap works differently starting python 3.9
# In previous versions, it expects all maps to be dictionaries, whereas starting py3.9
# It transforms list/tuples into a dict whose keys are the different elements.
arguments = ChainMap(dict.fromkeys(args), kwargs.values())
request_handler = None
for arg in arguments:
if isinstance(arg, RequestHandler):
request_handler = arg
break
if should_be_ratelimited(request_handler.request, area):
raise error.RatelimitError()
return func(*args, **kwargs)
@wraps(func)
def inner(self, *args, **kwargs):
if should_be_ratelimited(self.request, area):
raise error.RatelimitError()
return func(self, *args, **kwargs)

Copy link
Contributor Author

@shtlrs shtlrs Jan 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I first had it like that, by precising that the first argument is instance of RequestHandler, but then some tests failed because it got other unexpected arguments, so i just i thought I'd make it generic.

I prefer the explicitness of the parameter as well, so a combination like the one you're suggesting should work.

I'll try that out

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the forcepush 002b21f

Copy link
Contributor

@wookie184 wookie184 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, one thing.

src/pinnwand/handler/api_curl.py Show resolved Hide resolved
Copy link
Contributor

@wookie184 wookie184 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and works nicely. Seems like a good improvement to me

@supakeen
Copy link
Owner

supakeen commented Mar 2, 2024

Awesome stuff. Let's get the conflicts resolved and merge :)

The ratelimit_endpoint decorator will wrap any request handler's function to handle the ratelimit for it.

This avoids having the ratelimit checks being done in the handler's callback function's body
@shtlrs
Copy link
Contributor Author

shtlrs commented Mar 10, 2024

Awesome stuff. Let's get the conflicts resolved and merge :)

Rebased the commits, so this is done :D

@supakeen
Copy link
Owner

@shtlrs Awesome!!

@supakeen supakeen merged commit db64a8a into supakeen:master Mar 11, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants