Skip to content

Vector of length 0 as missing value for vector type columns is problematic #1990

@yaeldekel

Description

@yaeldekel

ValueMappingTransformer returns a vector of length 0 when the value it is trying to map is not in the dictionary. When data that has a missing value like this is saved as text data, then TextLoader is not able to reload this data. For example: if the mapping is
"a" -> 0,1
"b" -> 1,0
"c" -> 1,1

and we apply the ValueMappingTransformer to the following data:

a b
b c
d a

the result should be:

a b 0 1 1 0
b c 1 0 1 1
d a <missing vector of length 2> 0 1

but the actual result is
a b 0 1 1 0
b c 1 0 1 1
d a 0 1

which causes TextLoader to load the data as

a b 0 1 1 0
b c 1 0 1 1
d a 0 1 0 0

Metadata

Metadata

Assignees

Labels

P1Priority of the issue for triage purpose: Needs to be fixed soon.bugSomething isn't workingloadsaveBugs related loading and saving data or models

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions