textflint.input_layer.component.field.cn_text_field

CnTextField Class

class textflint.input_layer.component.field.cn_text_field.CnTextField(field_value, mask=None)

Bases: textflint.input_layer.component.field.field.Field

A helper class that represents input string that to be modified.

Parameters

• or list field_value (str) – the value of the field.

• mask (int) – mask label.

cn_processor = <textflint.common.preprocess.cn_processor.CnProcessor object>

ner()

ner fiction

Returns

ner tags

pos_tags()

pos tags fiction

Returns

ner tags

class textflint.input_layer.component.field.cn_text_field.CnProcessor(*args, **kwargs)

Bases: object

Text Processor class implement NER.

static tokenize(sent)

tokenize fiction

Parameters

sent (str) – the sentence need to be tokenized

Returns

list.the tokens in it

get_ner(sentence)

NER function.

Parameters

sent (str) – the sentence need to be ner

:return two forms of tags

The first is the triple form (tags,start,end) The second is the list form, which marks the ner label of each word such as 周小明去玩 [‘Nh’, ‘Nh’, ‘Nh’, ‘O’, ‘O’]

get_pos_tag(sentence)

pos tag function.

Parameters

sentence (str) – the sentence need to be ner

Returns

the triple form (tags,start,end)

class textflint.input_layer.component.field.cn_text_field.Field(field_value, field_type=<class 'str'>, **kwargs)

Bases: object

A helper class that represents input string that to be modified.

__init__(field_value, field_type=<class 'str'>, **kwargs)

Parameters

• field_value (string|int|list) – The string that Field represents.

• field_type (str) – field value type