v16 breaking changes

[josdejong]

Breaking changes planned for mathjs v16:

• Adopt reorganized config options ([BREAKING] chore: adopt proposed new config option organization #3592)
• uniformize bigint construction, see #3375
• Expose and systematize the parsing options and tokenizer configuration, making them mutable only in createed instances, see #3423
• Let the constructor functions like number() reckon with the option compute.uniformType, see #3451
• Move tolerance options to the log domain, see #3556.
• Change function setDistinct to not sort the results #3602
• Implement per-function history Documentation: Should function documentation (etc.) include indication of what version the feature was added #3341

[gwhitney]

Created the v16 branch, currently identical to develop, and submitted the first PR (#3592) in this series against it.

[gwhitney]

I would like to put in a request that for v16, we make a final consideration, decision, and implementation if decided in favor, of #3341 per-function History. The relevant quote from the second link is:

About the History description in the comments: I think it will be hard to maintain this by hand. Can we automate this? I think we can expand the script that extracts the documentation from the code comments to also collect all relevant git commits and add that to the generated web page of each function.

The reason I am bringing it up now is that the PRs in progress for v16 already touch a good fraction of the functions, so if there is something that needs to be done to each function source file, now is a good time, and I am game.

To your argument of "hard to maintain": Currently, on every PR merged, a line has to be added to the global HISTORY.md file. Switching to per-function History would just mean that a similar line would have to be added to each specific function file instead. So it's just at most one additional line change to each function file touched in a commit (and if there is no behavior change for that function, nothing needs to be added).

To your second point "we could collect all relevant git commits": If it were the case that the commit messages were in some uniform format, and annotated with the affected functions, and there were some straightforward way of getting the mathjs version from the commit, then I might agree. But the fact is the commit messages are all over the map and there is no clear way to tell whether a given commit affects any function or no function (even if you check whether a given source file is touched, it might just be a behavior-preserving re-factor). At best I think this would be significant coding with the the git API and the outcomes would be wonky and difficult to read; but more likely, I think it would be hopeless.

The pros of adopting per-function History are:

• A straightforward way of making feature history available in the mathjs.org documentation system; currently there is none -- you have to go to the source, and in particular the giant central HISTORY.md file. Sorry, I just realized that the giant HISTORY.md is transformed to the giant history.html webpage ;-)
• Makes history information more relevant and usable; it's currently difficulty to ferret out the information you might need in HISTORY.md because it so all-encompassing.
• Provides some way of getting information about the behavior of a function if you are not using the most current mathjs (an alternate idea for this would be to also/instead do a documentation PR, maybe to gh-pages, that would permanently archive the doc pages for each released mathjs version, and give you a dropdown at the top of the site that lets you choose the version you want to look at. That would be a welcome change, but it seems like more work, to me.)
• Provides "institutional memory" that for example could help prevent "feature oscillation" in which some aspect of a function's behavior could go back and forth as the original motivations for a change might be forgotten.
• Would help highlight the continual progress and evolution of mathjs.

The cons, so far as I can see, are solely the work of ensuring that in each PR that changes the behavior of a function, that information is recorded in some standard, recoverable way. Since only functions whose source files were touched can have changes that need recording, it seems to me that if the information is recorded in each such source file in some way, then this con is minimal. There are other schemes: adopting a careful, systematic format for the commit messages for each merge into develop,which would have to include an explicit list of the functions affected, so that the information really could be reconstructed from the git log. That happens to sound fussier to me, but your judgment might differ and in the end I am OK with any systematic way of retaining this information.

Anyhow, when the initial proof-of-concept implementation of your first suggested scheme (comments in the HISTORY section of the function definition) was introduced in PR #3375, you deemed that (a) it was a distraction from the main purpose of that PR, and (b) the feature needed more discussion. Since the transition to v16 will have a number of large-scope PRs anyway, this seems like the best opportunity we're going to have to adopt some scheme for this purpose and stop losing the information of when functions are introduced and how they change. So I'd like to conclude that discussion and then take the appropriate steps (or if the conclusion is that we won't adopt any per-function history then we can close #3341 and I can forget about the point). Thanks!

[josdejong]

Thinking about it again, I think automating this will be quite hard, and maintaining this by hand is very doable, though it is yet another task that needs to be done on every change. OK then let's go for it. I have to say, I do like libraries which have a clear history of any breaking changes or extensions of individual functions, it can be very useful.

On a side note: let's please discuss all changes themselves in their own topic, and keep this issue (#3508) to just discuss which changes to plan for in v16.

[gwhitney]

Ok, any further discussion of the details will be in #3341.

[gwhitney]

As far as I can see, there is nothing breaking about per-function history, so implementing it as a standalone PR against current develop.