Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Improve ] introduce multiprocessing in main.py for Scrapper-Service #6

Closed
rajatkb opened this issue Feb 22, 2020 · 5 comments · Fixed by #26
Closed

[Improve ] introduce multiprocessing in main.py for Scrapper-Service #6

rajatkb opened this issue Feb 22, 2020 · 5 comments · Fixed by #26
Assignees
@abhishekgupta368
Labels
enhancement New feature or request gssoc20 GSSOC label for gscco20 tag medium GSSOC label for beginner tag

Comments

@rajatkb
Copy link
Owner

rajatkb commented Feb 22, 2020 •
edited
Loading

The current implementation works in a sequential manner and is capable of using only one core of a multicore system. Something that should be leveraged.
REQUIREMENT

  • The Scrapper though may or may not do it's processed using threads or async should not affect launching the various Scrappers using MultiProcessing
@rajatkb rajatkb added enhancement New feature or request hard GSSOC label for beginner tag labels Feb 22, 2020
@rajatkb rajatkb added gssoc20 GSSOC label for gscco20 tag medium GSSOC label for beginner tag and removed hard GSSOC label for beginner tag labels Feb 23, 2020
@rajatkb rajatkb changed the title Introduce CLI & Multiprocessing in main.py for Scrapper-Service [Improve ] introduce CLI & Multiprocessing in main.py for Scrapper-Service Feb 28, 2020
@vipuldcoder
Copy link

I am interested with the issue.

@rajatkb rajatkb changed the title [Improve ] introduce CLI & Multiprocessing in main.py for Scrapper-Service [Improve ] introduce CLI in main.py for Scrapper-Service Mar 1, 2020
@rajatkb rajatkb changed the title [Improve ] introduce CLI in main.py for Scrapper-Service [Improve ] introduce multiprocessing in main.py for Scrapper-Service Mar 1, 2020
@abhishekgupta368
Copy link
Contributor

I would like to contribute.

@rajatkb
Copy link
Owner Author

rajatkb commented Mar 4, 2020

@abhishekgupta368 You interested to work on this ?

@abhishekgupta368
Copy link
Contributor

interested

@rajatkb
Copy link
Owner Author

rajatkb commented Mar 4, 2020

Assigning to @abhishekgupta368

@abhishekgupta368 abhishekgupta368 mentioned this issue Mar 5, 2020
Merged
@rajatkb rajatkb linked a pull request Mar 8, 2020 that will close this issue
added_multiprocessing #26
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abhishekgupta368 abhishekgupta368

Labels
enhancement New feature or request gssoc20 GSSOC label for gscco20 tag medium GSSOC label for beginner tag
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

added_multiprocessing abhishekgupta368/Conference-Notify
3 participants
@rajatkb @vipuldcoder @abhishekgupta368