[Uri] Paths with Unicode/UTF-8 incorrectly parsed/reported by System.Uri

[g0dsCookie]

Paths with Unicode/UTF-8 incorrectly parsed/reported by System.Uri

General

As described by the title, paths with Unicode/UTF-8 characters are incorrectly parsed/reported by System.Uri resulting in an invalid path. For example the path "/üri" will result in an Uri like "file:///%C3%BCri/üri" (note the unescaped /üri at the end).

This also happens with other Unicode/UTF-8 characters like £, §, etc. So you can replace ü by any other Unicode/UTF-8 character in my example and see the same result, e.g. the path is doubled.

Expected Result

PathAndQuery = AbsolutePath = "/%C3%BCri"
AbsoluteUri = "file:///%C3%BCri"

Results collected using mono 5.14 on the same Linux machine.

Actual Result

PathAndQuery = AbsolutePath = "/%C3%BCri" // This seems to be correct
AbsoluteUri = "/%C3%BCri/%C3%BCri" // Note the additional "/%C3%BCri" at the end
_string = "/üri/üri" // Note the additinal "/üri" at the end

Using the .NET core version mentioned below.

System Informations

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.1.302
 Commit:    9048955601

Runtime Environment:
 OS Name:     gentoo
 OS Version:
 OS Platform: Linux
 RID:         gentoo-x64
 Base Path:   /opt/dotnet_core/sdk/2.1.302/

Host (useful for support):
  Version: 2.1.2
  Commit:  811c3ce6c0

Code to reproduce

using System;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            var uri = new Uri("/üri");
            Console.WriteLine(uri.ToString()); // file:///%C3%BCri/üri
        }
    }
}

Above code prints "file:///%C3%BCri/üri" while "file:///%C3%BCri" is expected.

Using "/üri/üri" in the Uri ctor results in a path like "file:///%C3%BCri/%C3%BCri/üri/üri".

[g0dsCookie]

Also DOS-like and UNC Paths are treated correctly on windows.

c:\üri -> file:///c:/üri
//computer/üri -> file://computer/üri

[karelz]

@g0dsCookie let's not mix multiple problems in the same issue. I don't think Windows paths are recognized in Uri on Linux. That is IMO by design - @rmkerr can confirm.

Regarding your original report: Can you please clarify (please edit the top post) what is the actual result on each platform and what is expected?

[g0dsCookie]

@g0dsCookie let's not mix multiple problems in the same issue. I don't think Windows paths are recognized in Uri on Linux. That is IMO by design - @rmkerr can confirm.

Actually paths like "C:\üri" and "\localhost\üri" are correctly parsed on Linux with .NET Core.

Regarding your original report: Can you please clarify (please edit the top post) what is the actual result on each platform and what is expected?

I updated my original report. Hope it's clearer now. It seems like there's a problem when parsing UTF-8/Unicode characters which causes the path to be doubled for every UTF-8 character encountered.

[rmkerr]

Thanks for the detailed report @g0dsCookie. This looks really interesting. I'm not going to be able to take a look at it immediately, but I think we should try to get this fixed for 3.0.

[wtgodbe]

@tarekgh does this look like something that might be related to libicu?

[tarekgh]

@wtgodbe I cannot tell without looking :-) does this issue not repro on Windows?

[wtgodbe]

It doesn't repro on Windows, I get the same results as https://github.com/dotnet/corefx/issues/33557#issuecomment-439075084

[tarekgh]

It looks on Windows this is by design as the Uri cannot start with one '/' char.

https://github.com/dotnet/corefx/blob/master/src/System.Private.Uri/src/System/Uri.cs#L3737

and on Linux this can be a valid Uri according to https://github.com/dotnet/corefx/blob/master/src/System.Private.Uri/src/System/Uri.cs#L3663

I didn't look at Linux yet to know why we are returning the result we are seeing here.

[tarekgh]

I have looked at the issue on Linux, the problem has nothing to do with icu. here is what is the problem:

The URI code detect that running on Linux and the string starts with '/' which means it could be a valid file path. and store the internal uri._string as the original value "/üri". Later, the code will call the method ParseRemaining which will call EscapeUnescapeIri.

https://github.com/dotnet/corefx/blob/f5b57382cd5fef53cf09e5fd6b9b9812dfddb953/src/System.Private.Uri/src/System/Uri.cs#L3394

EscapeUnescapeIri will return "/üri" and then code will concatenate this value to the original _string. that means _string now will be storing "/üri/üri"

Then later the code will try to get the host name. will detect the host name should be the first 4-characters "/üri" and will call EscapeString helper method to normalize this name which will return "/%C3%BCri"

https://github.com/dotnet/corefx/blob/f5b57382cd5fef53cf09e5fd6b9b9812dfddb953/src/System.Private.Uri/src/System/Uri.cs#L2520

that makes the whole uri as "file:///%C3%BCri/üri"

Let me know if I can help in anything more.