textflint.engine

textflint Engine Class

class textflint.engine.Engine

Bases: object

Engine class of Text Robustness.

Support run entrance which automatically finish data loading, transformation/subpopulation/attack generation and robustness report generation.

Also provide interfaces of each layer to practitioners.

run(data_input, config=None, model=None)

Engine start entrance, load data and apply transformations, finally generate robustness report if needed.

Parameters

• data_input (dict|list|string) – json object or json/csv file

• config (string|textflint.Config) – json file or Config object

• model (textflint.FlintModel) – model wrapper which implements FlintModel abstract methods, not a necessary input.

Returns

save generated data to out dir and provide report in html format.

load(data_input, config=None, model=None)

Load data input, config file and FlintModel.

Parameters

• data_input (dict|list|string) – json object or json/csv file

• config (string|textflint.Config) – json file or Config object

• model (textflint.FlintModel) – model wrapper which implements FlintModel abstract methods, not a necessary input.

Returns

textflint.Dataset, textflint.Config, textflint.FlintModel

generate(dataset, config, model=None)

Generate new samples according to given config, save result as json file to out path, and evaluate model performance automatically if provide model.

Parameters

• dataset (textflint.Dataset) – container of original samples.

• config (textflint.Config) – config instance to control procedure.

• model (textflint.FlintModel) – model wrapper which implements FlintModel abstract methods, not a necessary input.

Returns

save generated samples to json file.

report(evaluate_result)

Automatically analyze the model robustness verification results and plot the robustness evaluation report.

Parameters

evaluate_result (dict) – json object contains robustness evaluation result and other additional information.

Returns

open a html of robustness report.

textflint.engine.auto_config(task='UT', config=None)

Check config input or create config automatically.

Parameters

• task (str) – task name

• config (str|dict|textflint.config.Config) – config to control generation procedure.

Returns

textflint.config.Config instance.

textflint.engine.auto_dataset(data_input=None, task='UT')

Create Dataset instance and load data input automatically.

Parameters

• data_input (dict|list|string) – json object or json/csv file.

• task (str) – task name.

Returns

textflint.Dataset instance.

textflint.engine.auto_flintmodel(model, task)

Check flint model type and whether compatible to task.

Parameters

• model (textflint.FlintModel|str) – FlintModel instance or python file path which contains FlintModel instance

• task (str) – task name

Returns

textflint.FlintModel

textflint.engine.auto_generator(config_obj)

Automatic create task generator to apply transformations, subpopulations and adversarial attacks.

Parameters

config_obj (textflint.Config) – Config instance.

Returns

textflint.Generator

textflint.engine.auto_report_generator()

Return a ReportGenerator instance.

Returns

ReportGenerator