MDEB data sets

[andybeet]

We agreed that a good option to integrate these MDEB datasets into the package would be to set up a cron job to check for changes and add as sf objects. A few questions:

1. How should we name these objects? Some of the titles seem a little too long
2. How should we deal with the documentation of the data sets? A boiler plate file (with little info, all datasets looking the same) pointing to the MDEB site?
3. Cron job on a weekly or monthly schedule?

@jmhatch @kimberly-bastille

[andybeet]

So i have used the name field of the mdeb data objects as a starting point for the sf object name

eg. ORIGINAL NAME -> new name
SMIT_TURTLE_STRATA -> turtle_strata
SMIT_SHRIMP_STRATA -> shrimp_strata

Does this seem ok?
I have all of the data sets added to the package locally. Should i add those to the PR so they are in the repo BEFORE we start a github workflow on a cron job?

I have a template for a workflow set up also. you can see it in the PR

Comments/suggestions?

[jmhatch]

@andybeet @kimberly-bastille

• I think the naming scheme you came up with is good
• I think it makes sense to add the mdeb datasets to the PR so it's available before the github action takes effects on a cron job (but probably doesn't matter much?)
• I think documentation should be consistent throughout, happy to develop a template that could be used or you may have one (I didn't dig through your PR yet)
• I would say monthly updates for mdeb (at this point), survey strata / footprints shouldn't change that frequently (monthly might be too much, but that could change if alternate data is hosted)

Thanks!

[andybeet]

ok. i will add the data sets.

Re documentation: i dont have anything yet and would welcome a template.
If we pull from mdeb on some schedule and new data has been added, would we then add the help file manually or do you think we could generate it from the pull?

I'm ok with monthly too

[andybeet]

And the naming scheme i used actually overwrites 3 of the data sets we have deprecation warnings for:
bts_strata, shrimp_strata, and ecomon_strata

How should we handle that?
Just overwrite and push?

[jmhatch]

OK, I'll work on a template for data documentation. Should be straightforward once the data is pulled and in the data/ dir. I'll write an R function that will take a dataset name and then spit out an Roxygen file that can then be built, filling in the necessary bits with the info from the data set.

It'd be good to keep the data that would get overwritten for now, at least until the deprecation warnings are pushed to main so that users get a warning about the change. Looks like you kept mdeb data names lowercase, and the pre-existing data names are upper? I think that's a good compromise.

Do you think it would be good to add the new data names to the old data documentation somewhere? So users have a re-direct.

[andybeet]

Thanks Josh, (for the documentation)
So i thought it was a good idea to keep new names (mdeb) all lower case but on my windows machine they are not recognized as different from the "old" (mixed case) names. Not sure what to do about that.

[jmhatch]

Bummer about the names not being case sensitive, that would of been a nice solution.

For the data documentation, I created a new R function in data-raw/generate_roxygen_mdeb.R. I then added some code to your existing workflow data-raw/get_mdeb_for_package.R, which calls generate_roxygen_mdeb() after the data are saved. Might need to re-run with overwrite=TRUE to generate the data docs.

Just pushed both to andy_mdebdb for you to take a look.

[andybeet]

This is awesome!! Works like a charm!!

Next question. We currently have these data sets in the reference section separated by type, eg. survey, fishing area, closed area, protected species. Do we want to keep that? If so i can add these new data sets tot the yml in the appropriate spot (they are all surveys right?)

[jmhatch]

That makes sense to me. Yes, all surveys at the moment.