You will get a faster and better answer if the people reading your question can reproduce your problem without guessing. The Q&A categories pre-fill a template with these headings, and here is what to put under each.
What are you trying to do? The goal, not only the error. “Get the price and availability of every product in a category” lets someone suggest a better way. “Fix a 520” does not.
What did you try? The request or spider code, as text. For Zyte API, that is the JSON body you send, with your API key removed. For Scrapy, the relevant spider code and the settings that matter, such as the scrapy-zyte-api settings.
What happened? The exact error, status code or output. For Zyte API, include the whole error response, for example {"status": 520, "type": "/download/temporary-error"}, and say how often it happens. Always, or sometimes, makes a difference.
Versions. Python, Scrapy, scrapy-zyte-api or python-zyte-api, and where it runs, for example locally or on Scrapy Cloud.
A few things that help.
- Give the URL you are targeting, or a representative one. If it is private, say what kind of site it is.
- Reduce the problem to the smallest request or spider that still fails.
- One problem per topic. If you have three, post three.
- When someone’s reply solves it, mark it as the solution, and say what the fix was if you found it yourself.
Here is an example of a question that gets answered quickly.
What are you trying to do? Get product names and prices from a category page on a shop that renders prices with JavaScript.
What did you try?
{"url": "https://example.com/category/shoes", "httpResponseBody": true}. The prices are missing from the HTML.What happened? The response is a 200 and the price elements are empty.
Versions. Python 3.12, python-zyte-api (latest).
That question has a goal, the exact request, a precise symptom and the versions, so someone can answer it in one reply (try browserHtml instead of httpResponseBody, since the prices are rendered by JavaScript).