Can I send my own User-Agent header through Zyte API, and what happens with large pages?
Asked and answered by the Zyte team, as a reference for the community.
User-Agent. On HTTP requests you can set it with the request headers, but Zyte API may replace your value on some websites when that is needed to avoid bans. On browser requests you cannot override it.
If you want exactly the response that curl or another non-browser client gets, setting the header may get you there. Keep in mind that Zyte API aims to return what a real browser would get, so a response that only a non-browser client sees may not be reachable, because some sites can tell the two apart.
Size. The limit for browserHtml and httpResponseBody is 10 MB, measured before base64 encoding. Longer responses are truncated, and HTTP compression does not change the limit.
Docs: Zyte API frequently asked questions - Zyte documentation