Evaluator integration ideas: Logging, config

[benjaminy]

Here are a couple of convenience features to consider:

Logging. It would be nice if the evaluator could put notes of some kind in the log for reference after a run is done. This could be implemented by passing the logging object to the evaluator function. Or the evaluation result could include an additional field like notes that just gets written out to the log.

Config. It would be nice to do certain kinds of experiments with different evaluation tweaks without changing the evaluation code itself. This could be accomplished by passing the config object to the evaluator function. Then one could tweak the config values in one place.