normalisation of urls containing non-ascii domains is broken and loses data

[wbolster]

Initial parsing works:

>>> rfc3986.uri_reference('http://æåëý.com/path?query#fragment')
URIReference(scheme='http', authority='æåëý.com', path='/path', query='query', fragment='fragment')

Subsequent normalisation silently loses data:

>>> rfc3986.uri_reference('http://æåëý.com/path?query#fragment').normalize()
URIReference(scheme='http', authority=None, path='/path', query='query', fragment='fragment')

[sigmavirus24]

Correct. We do not yet handle IRIs. (RFC 3987)

[wbolster]

Fwiw, preprocessing by replacing the host name part with its IDNA-encoded (xn--…) equivalent using the url parsing routines from the urllib3 package, before passing it to uri_reference() sort of "works" as a work-around.

[pombredanne]

Correct. We do not yet handle IRIs. (RFC 3987)

@sigmavirus24 the doc also state the same "There's presently no support for IRIs as defined in RFC 3987." .... but is this correct? I can see some IRI support in the code proper.

[sigmavirus24]

@pombredanne yes there was some IRI support. I've been away from this library long enough to need to dig in and figure out whether this issue is still valid. Unfortunately I have almost zero time for these projects these days