Only log certain metrics with spacy.MLflowLogger.v2

[ryuzakinho]

I am using the spacy.MLflowLogger.v2 logger in spaCy while training a textcat model. Here's how my logger config looks like:

[training.logger]
@loggers = "spacy.MLflowLogger.v2"
experiment_id = "1"
run_name = "somemetrics"
nested = False
remove_config_values = ["paths.train", "paths.dev", "corpora.train.path", "corpora.dev.path"]

The logging works well, however, it is logging metrics I do not plan to track and I would therefore only like to track a couple of metrics e.g. cats_macro_f and cats_micro_f.

I have tried using the log_custom_stats = ["cats_micro_f"] option, however, it still logs all the other metrics. In fact, when looking at the logger source code, I can see the following function:

def log_step(info: Optional[Dict[str, Any]]): 
    _log_step_mlflow(mlflow, info) 
    _log_custom_stats(mlflow, info, match_stat)

Which will anyway log all metrics in _log_step_mlflow before using the regular expressions listed in log_custom_stats.

Could anyone please explain how I prevent spacy.MLflowLogger.v2 from logging all metrics and only log specific metrics?

[rmitsch]

Hi @ryuzakinho, thanks for reporting this. I'll look into this. In the meantime you can register your own logger that doesn't write the stats regardless, if you absolutely do not want them in your MLFlow experiment setup. You can have a look at the logger implementation as a starting point.

[ryuzakinho]

Thanks @rmitsch . Please find some background behind my need below:

I have a text classification task where classes would be named e.g. 'remote support (9-17)'. While spaCy's textcat accepts such class names. The MLFlow logger does not like characters like '(' in metric names and therefore breaks.

[rmitsch]

I suggest replacing the problematic characters as a workaround.