Guidelines for contributors

[jpaillard]

Can we start listing guidelines for contributors? We recently encountered cases where alignment could have saved some back and forth:

• Testing: tests should be implemented in functions; try to avoid classes.
• Randomness/seeding: Methods should allow for obtaining reproducible and repeatable results. The randomness should be controlled by the random_state argument at instantiation. When default arguments are sklearn estimators, they should be seeded.
• Methods with randomness should be tested for reproducibility
• ...

[lionelkusch]

This is already started in #307 and I let some issues open because the consensus on it should be written in some documentation.
However, @bthirion says that this is not the priority. Due to this, I haven't kept it updated since last the beginning of the summer.
Moreover, @bthirion seems to prefer not to take decisions for the moment.

[bthirion]

I think that such guidelines have to come from a consensus. The first step was thus to build the consensus ...
I'm happy to start drafting them (I think we agree on the above propositions), but indeed, we're not ready yet to develop them much.
One possibility is to refer to other guidelines, e.g. sklearn guidelines (provided we're happy with those).

[lionelkusch]

You should look at my work in PR #307 is a proposition based on multiple guidelines.

@jpaillard don't hesitate to complete it. If you want, I can create a branch on mind-inria to facilitate the work.

[jpaillard]

I think it's great forward-looking, and the kind of long-term guidelines we want to have.
But what I was referring to is more of a "minimalist set of guidelines", mostly for us at the moment, so that each time we agree on a rule (examples listed above), we write it down and we can refer to it.

[lionelkusch]

I think that this can be included in the documentation for developers.
However, I don't have a clear idea of formatting them and including them in the library.
Scikitlearn propose one page documentation with a different section but I don't like it.

If you have any idea of the format and the organisation of this type of "minimalist set of guidelines", there are welcome.