textflint.generation_layer.transformation.CWS.swap_verb

Replace verb with detachable words or a one a.

class textflint.generation_layer.transformation.CWS.swap_verb.SwapVerb(**kwargs)[source]

Bases: textflint.generation_layer.transformation.transformation.Transformation

Replace verb with detachable words or a one a.

__init__(**kwargs)[source]
Parameters

  • detachable_word_list (list) – word can be replaced detachable word dictionary

  • AoneA_list (list) – word can be replaced by a one a dictionary

static get_pos_tag_list(pos_tag)[source]

get the list of pos tag

Parameters

pos_tag (list) – transform tuple to list

static check_part_pos(sentence)[source]

get the pos of sentence if we need

Parameters

sentence (str) – origin word

Returns

bool

class textflint.generation_layer.transformation.CWS.swap_verb.CnProcessor(*args, **kwargs)[source]

Bases: object

Text Processor class implement NER.

static tokenize(sent)[source]

tokenize fiction

Parameters

sent (str) – the sentence need to be tokenized

Returns

list.the tokens in it

get_ner(sentence)[source]

NER function.

Parameters

sent (str) – the sentence need to be ner

:return two forms of tags

The first is the triple form (tags,start,end) The second is the list form, which marks the ner label of each word such as 周小明去玩 [‘Nh’, ‘Nh’, ‘Nh’, ‘O’, ‘O’]

get_pos_tag(sentence)[source]

pos tag function.

Parameters

sentence (str) – the sentence need to be ner

Returns

the triple form (tags,start,end)

class textflint.generation_layer.transformation.CWS.swap_verb.Transformation(**kwargs)[source]

Bases: abc.ABC

An abstract class for transforming a sequence of text to produce a list of potential adversarial example.

processor = <textflint.common.preprocess.en_processor.EnProcessor object>
transform(sample, n=1, field='x', **kwargs)[source]

Transform data sample to a list of Sample.

Parameters

  • sample (Sample) – Data sample for augmentation.

  • n (int) – Max number of unique augmented output, default is 5.

  • field (str|list) – Indicate which fields to apply transformations.

  • **kwargs (dict) –

    other auxiliary params.

Returns

list of Sample

classmethod sample_num(x, num)[source]

Get ‘num’ samples from x.

Parameters

  • x (list) – list to sample

  • num (int) – sample number

Returns

max ‘num’ unique samples.

textflint.generation_layer.transformation.CWS.swap_verb.download_if_needed(folder_name)[source]

Folder name will be saved as .cache/textflint/[folder_name]. If it doesn’t exist on disk, the zip file will be downloaded and extracted.

Parameters

folder_name (str) – path to folder or file in cache

Returns

path to the downloaded folder or file on disk

textflint.generation_layer.transformation.CWS.swap_verb.plain_lines_loader(path)[source]

read data