Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: wrap generate_keys() in future #168

Merged
merged 8 commits into from
Nov 28, 2023

Conversation

RahulDubey391
Copy link
Contributor

@RahulDubey391 RahulDubey391 commented Nov 23, 2023

Added the following changes to the files:

  • Added Async Definition to the generate_key() in utils.py
  • Added wrap_future for class member declaration in connector.py

Fixes #14

@RahulDubey391 RahulDubey391 changed the title Adding Async Wrap for generate_key() feat: Adding Async Wrap for generate_key() Nov 23, 2023
@RahulDubey391 RahulDubey391 reopened this Nov 23, 2023
@jackwotherspoon
Copy link
Collaborator

Hi @RahulDubey391! Thanks for the PR 😄

It still needs a few things to be fully complete. You will need to update your code to add an await where the keys are used to actually await the future reference.

You will also want to pip install nox on your local machine in order to run the unit tests.

The command to run the unit tests is nox -s unit-3.11 where you can change the python version (3.11) depending on your environment. You can also run the linter by runnng nox -s lint and see its suggestions.

I have added the await where ever the key generation is happening, but after running the unit tests, the code is failing. I am pushing the changes although, will keep on testing until the tests are passed.
@jackwotherspoon
Copy link
Collaborator

@RahulDubey391 please take a look at failing tests and lint jobs.

Check my previous comment for running these locally and making sure they work. If this is too much trouble I am happy to jump in and help finish off this PR for you 😄

Thanks again, we really appreciate the contribution

@RahulDubey391
Copy link
Contributor Author

Hi @jackwotherspoon, thanks for helping me out! After your suggestions, I re-ran the test cases. Lint Checks have passed already. For Unit Test cases, I am facing issue with 3 particular cases. 20/23 have passed already. I need your help in other 3 cases. These cases were:

  • test_schedule_refresh_wont_replace_valid_result_with_invalid()
  • test_schedule_refresh_expired_cert()
  • test_seconds_until_refresh_under_4_mins()

Can you guide me around how can I fix the above?

Also for the System Test cases, I am getting the "ALLOYDB_INSTANCE_IP" environment variable KeyError.

@jackwotherspoon
Copy link
Collaborator

@RahulDubey391 Looks like tests are all green actually 👍 🟢

I will give the PR a final look over on Monday morning, otherwise we should be good to go ahead and merge. Thanks again!

@RahulDubey391
Copy link
Contributor Author

@RahulDubey391 Looks like tests are all green actually 👍 🟢

I will give the PR a final look over on Monday morning, otherwise we should be good to go ahead and merge. Thanks again!

Thanks for the help and guidance @jackwotherspoon, much appreciated!

Copy link
Collaborator

@jackwotherspoon jackwotherspoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple small nits to look at cleaning up, let me know if you want me to take care of them as you have done an awesome job and I can take over if its too much trouble.

@@ -53,7 +54,7 @@ def __init__(
self,
instance_uri: str,
client: AlloyDBClient,
keys: Tuple[rsa.RSAPrivateKey, str],
keys: asyncio.Future,
Copy link
Collaborator

@jackwotherspoon jackwotherspoon Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can try to type hint the return of the future? I believe this was introduced in Python 3.8 so we should be okay.

Suggested change
keys: asyncio.Future,
keys: asyncio.Future[Tuple[rsa.RSAPrivateKey, str]],

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jackwotherspoon , thanks for the feedback! I have pushed the above change in the latest push

google/cloud/alloydb/connector/instance.py Outdated Show resolved Hide resolved
Comment on lines 34 to 37
event_loop = asyncio.get_running_loop()
keys = asyncio.wrap_future(
asyncio.run_coroutine_threadsafe(generate_keys(), event_loop), loop=event_loop
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just await this directly here...? The reason we have to use the future in the main code is because the loop is running in a separate thread. The tests run in the same thread so I think we can just await it directly.

Suggested change
event_loop = asyncio.get_running_loop()
keys = asyncio.wrap_future(
asyncio.run_coroutine_threadsafe(generate_keys(), event_loop), loop=event_loop
)
keys = await generate_keys()

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for other test cases

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jackwotherspoon , I have removed the separate Async loops.

But one doubt, since the keys are being passed to the Instance object as future, due to double await, the Tests are throwing error that "an awaited expression can't be awaited again". So I removed the await for the generate_keys() function so finally it's being awaited once in the main Instance loop.

Tell me if it's fine otherwise we can revert it back.

@jackwotherspoon jackwotherspoon changed the title feat: Adding Async Wrap for generate_key() feat: wrap generate_keys() in future Nov 27, 2023
@RahulDubey391
Copy link
Contributor Author

Couple small nits to look at cleaning up, let me know if you want me to take care of them as you have done an awesome job and I can take over if its too much trouble.

Hi @jackwotherspoon , thanks a lot! Sure if you can take over the last issue with Async tests failing in some cases, that would be much helpful. But do let me know what works best, I am also pushing my changes.

@jackwotherspoon
Copy link
Collaborator

Sure if you can take over the last issue with Async tests failing in some cases, that would be much helpful.

Done! Made the tests use asyncio.Task objects which can be awaited multiple times. 😄

Will go ahead and merge this, thanks again for the contribution! 👏

@jackwotherspoon jackwotherspoon merged commit 964deb0 into GoogleCloudPlatform:main Nov 28, 2023
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Wrap key generation in future
2 participants