Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document how to handle action failures #157
base: main
Are you sure you want to change the base?
Document how to handle action failures #157
Changes from 9 commits
7cc483b
47197ee
00b2c2a
20f3715
deedfff
75faf68
cdc0aa1
f19e157
62c9ae6
ea2da8f
e1eb2b1
12c7907
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a lot of code in this section; I wonder if we should make it a part of the library itself instead.
It seems like a general theme: if action failed, should the response be considered a success or a failure? The current default is success, but it seems that making it a failure is better (with an option to get back), as it allows to detect more issues.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
deedfff does this for the middleware. Do we want to do something about the cache policy, or is it OK to keep that documentation-only?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any interaction with the ZYTE_API_ACTION_ERROR_HANDLING / ZYTE_API_ACTION_ERROR_RETRY_ENABLED settings? E.g. do you need this for all values option values? Or do some of these option values require caching to work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess you only need this for the default handing,
"pass"
, as I don’t imagine caching caches exceptions. But I will test this out manually.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was wrong.
I tested all 6 scenarios, default policy/custom policy, pass/ignore/err, and pass/ignore/err make no difference, with the default policy responses are cached, with the custom policy they are not. Not sure if the current docs need any clarification, given the custom policy is relevant regardless of those settings.
Test details
I cloned https://github.com/zytedata/zyte-spider-templates-project and added 2 files:
And then tried all 6 scenarios.
Default cache policy, default action error handling (
"pass"
), responses are cached:Default cache policy,
"ignore"
action error handling, responses are cached:Default cache policy,
"err"
action error handling, responses are cached:Custom cache policy, default action error handling (
"pass"
), responses are not cached:Custom cache policy,
"ignore"
action error handling, responses are not cached:Custom cache policy,
"err"
action error handling, responses are not cached:There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I’m not sure I fully understood your question, though, I might be missing some angle)
Check warning on line 81 in scrapy_zyte_api/_middlewares.py
Codecov / codecov/patch
scrapy_zyte_api/_middlewares.py#L79-L81
Check warning on line 92 in scrapy_zyte_api/_middlewares.py
Codecov / codecov/patch
scrapy_zyte_api/_middlewares.py#L92
Check warning on line 96 in scrapy_zyte_api/_middlewares.py
Codecov / codecov/patch
scrapy_zyte_api/_middlewares.py#L96
Check warning on line 98 in scrapy_zyte_api/_middlewares.py
Codecov / codecov/patch
scrapy_zyte_api/_middlewares.py#L98