Generalised Madrigal Instrument

[aburrell]

The new general methods for Madrigal data that uses MadrigalWeb makes it possible to create a general instrument object for any data stored there. Users could supply the instrument and data code at instantiation instead of a name, and the only thing that would be lacking that the specific instruments have are targeted acknowledgements and the clean routine. Using the Madrigal instrument and experiment codes could be made easier by having the general madrigal init routine could grab these keyword arguments, use them along with functools.partial to set the load and download routines as needed.

[rstoneback]

See jroisr branch for work in this area.

[rstoneback]

Almost everything needed to close this issue is supported except for one thing, filenames. We need an automated way to get the filename format string. Perhaps we could just require that from the user?

[aburrell]

MadrigalWeb has a routine that will list out the file names for a specific time period and experiment, so I don't think we should require that of the user.

[rstoneback]

I tried one of those routines (perhaps the wrong one) and it seemed to include a number of filename options not actually present in the filenames I downloaded. The issue is teaching pysat how to parse the filenames.

Current support keys in on the time only, which is the most relevant value pysat needs. It isn't very specific though. There is a potential collision if the general support is used for more than one instrument, all sets of files will currently be picked up.

[aburrell]

Hmmm, maybe we can ask the user to do this, but also provide a guide on how to use MadrigalWeb to get this information.

[rstoneback]

I think a guide is a good way to go. We can download the files without knowing the filename specifics and a user can easily translate a given filename to a format template string. They could pass it in at Instrument instantiation, inst = pysat.Instrument('madrigal', 'pandas', madrigal_inst_code=8100, madrigal_tag=10241, file_format='dmsp_ivm_{year:02d}_{moth:02d}.....', )

I'll need to tweak the general madrigal code to fully support this (oops). Instantiation may be a little verbose but with the file_format string in there it will be as robust as any other pysat Instrument.

[rstoneback]

Code was already in good shape. I added to the docstring to clarify support.

[jklenzing]

Note from pysat/pysat#175: mad_methods.download not implemented via functools.partial. Could be updated for consistency as part of generalized madrigal instrument.