Skip to content

Handle ISO-8859-9 encoding while fetching HTTP response #1045

@nikneym

Description

@nikneym

You may reproduce via lightpanda-io/demo#69.

The HTML parsing failing if Content-Type is given ISO-8859-9.

* Request completely sent off
< HTTP/2 200
< content-type: text/html; charset=ISO-8859-9
< content-security-policy: object-src 'none';base-uri 'self';script-src 'nonce-Q6q-_mo0CXrGYWoedX1mGg' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/xsrp
< accept-ch: Sec-CH-Prefers-Color-Scheme
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< content-encoding: gzip
< date: Fri, 12 Sep 2025 11:42:39 GMT
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< expires: Fri, 12 Sep 2025 11:42:39 GMT
< cache-control: private
< set-cookie: NID=525=eMD6244YFmPpoAbacf5t97406d-9mOAmAqqe3F67zHCogH4vwWhGbMaJ7SWOFUR2KC1KXvh0RW_wRR76KPoATQvMG-Hb9nJD1Uo6D9PxSZlUxQQvQqMSsIQ5jdsq97695Xrcd1Vz6UmGdxqSgvKe9YuvbS5ewVWn81JipwQJGWUtf257DiB5v-awDJux_8JKs2h_B7kgSKU806AoKd4; expires=Sat, 14-Mar-2026 11:42:38 GMT; path=/; domain=.google.com; HttpOnly
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
<
ERROR http : data_callback . . . . . . . . . . . . . . . . .  [+5598ms]
      err = BadEncoding
      req = GET https://www.google.com/search?ie=ISO-8859-1&hl=tr&source=hp&biw=&bih=&q=lightpanda&iflsig=AOw8s4IAAAAAaMQVPg2w3BwseditS2s1ZbjXM82_cZLX&gbv=2

* client returned ERROR on write of 2330 bytes
* Connection #1 to host www.google.com:443 left intact
ERROR http : navigate failed . . . . . . . . . . . . . . . .  [+5598ms]
      err = WriteError

Segmentation fault at address 0x0
bindings/hubbub/parser.c:1010:2: 0x7a7b41d in dom_hubbub_parser_destroy (bindings/hubbub/parser.c)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions