ReadFromTextFile cannot load data as KeyType

[rogancarr]

CreateTextLoader can read in data as KeyTypes, but ReadFromTextFile cannot.

CreateTextLoader works like this:

new TextLoader.Column(matrixColumnIndexColumnName, DataKind.U4, new [] { new TextLoader.Range(1) }, new KeyCount(20)),

Now, there is a KeyType(Count = 20) attributes that you can put on a class property, but this only seems to be used by scoring. I'd like to be able to use that, or something like that, to read in data directly as KeyType.

[rogancarr]

This isn't quite a bug, and it's definitely not necessary for v1.0, but it's a clear mismatch in the API, and the KeyType attribute led me astray for quite a bit, so if it's not a v1.0 fix, I'd recommend we target this for v1.1.

[eerhardt]

@codemzs - This was the issue that @nicolehaugen ran into. I think this would be a good "first time contributor" type of issue.

[Pomoinytskyi]

Hi all. I'd like to fix this issue.
Is it still actual.
I cannot find any references for ReadFromTextFile in master branch.
In what branch it should be fixed?

[eerhardt]

@Pomoinytskyi - Yes this is still a valid issue. ReadFromTextFile was renamed to LoadFromTextFile. See

machinelearning/src/Microsoft.ML.Data/DataLoadSave/Text/TextLoaderSaverCatalog.cs

Lines 148 to 154 in a072b5e

	public static IDataView LoadFromTextFile<TInput>(this DataOperationsCatalog catalog,
	string path,
	char separatorChar = TextLoader.Defaults.Separator,
	bool hasHeader = TextLoader.Defaults.HasHeader,
	bool allowQuoting = TextLoader.Defaults.AllowQuoting,
	bool trimWhitespace = TextLoader.Defaults.TrimWhitespace,
	bool allowSparse = TextLoader.Defaults.AllowSparse)

Basically, the issue here is that you can't load KeyType data from a file when you use the CreateTextLoader and LoadFromTextFile overloads that take a Type to define the column information.

This method:

machinelearning/src/Microsoft.ML.Data/DataLoadSave/Text/TextLoader.cs

Lines 1444 to 1451 in a072b5e

	internal static TextLoader CreateTextLoader<TInput>(IHostEnvironment host,
	bool hasHeader = Defaults.HasHeader,
	char separator = Defaults.Separator,
	bool allowQuoting = Defaults.AllowQuoting,
	bool supportSparse = Defaults.AllowSparse,
	bool trimWhitespace = Defaults.TrimWhitespace,
	IMultiStreamSource dataSample = null)
	{

Should be respecting the KeyTypeAttribute.

Check out the referenced duplicate issue - #3811 as well for more information.

[aslotte]

@eerhardt I'm planning on opening a PR for your review for this issue this weekend. Would much appreciate your guidance if I'm on the right track as this would be my first issue. Thanks!