[discussion]: How to implement DST paths as project IDs

[signekb]

Description

Marie sent me the following overview of paths on DST (removed some ID's):

Project	Grunddata (DST)	Eksterne data (SDS)
Project DB	E:\ProjektDB\SDCA\<project_id>_DST_db_rawdata\Grunddata	E:\ProjektDB\SDCA\<project_id>_DST_db_rawdata\Eksterndata
Darter	E:\rawdata\<project_id>\Grunddata	E:\rawdata\<project_id>\Eksterne data
DP-Next*	E:\ProjektDB\SDCA\Rawdata\<project_id>\Grunddata	E:\ProjektDB\SDCA\Rawdata\<project_id>\Eksterne data
Marie's**	E:\ProjektDB\SDCA\<maries_id>_db_manager\Projekter\<project_id>\Grunddata	E:\ProjektDB\SDCA\<maries_id>_db_manager\Projekter\<project_id>\Eksterne data

*And other---newer---projects on DST.
**These are the path to folders where Marie works with the data to make subpopulations of project db data for different projects

Can we find a nice way to accommodate these different paths in the functions in this package?

[lwjohnst86]

oooo, that's complicated 🤔 I think the Project DB and Marie's folders are custom folders they made, not that comes with DST. So many most of the core functions (convert, read) require an absolute path, and then we have several helper functions to help with giving paths that are e.g. like in DARTER one or in DP-Next one.

[lwjohnst86]

Could have an interface like..?

list_project_files("ID") |>
  get_register_path("NAME") |>
  read_register()

list_projectdb_files("ID") |>
  get_sas_paths() |>
  convert_...()

Or something that makes it easier to get and wrangle paths based on the Project ID. Or even list_project_files(id) (I assume IDs are unique? 🤔 )

[signekb]

Yes, I think that something like that might be the way to go for us 👍

So, we could have some---quite specific---helper functions along the lines of:

• get_sdca_projectdb_path(project_id, data=c("grunddata", "eksterne"))
• get_darter_path(project_id, data=c("grunddata", "eksterne"))
• get_sdca_project_path(project_id, data=c("grunddata", "eksterne"))
• get_sdca_dbmanager_path(project_id, dbmanager_id, data=c("grunddata", "eksterne")

EDIT: I don't particularly like the name of the data argument. It could also be called source? Or even be something like source=c("dst", "sds"). But maybe using "grunddata" and "eksterne" is closer to the path itself and therefore more intuitive. I think that I prefer source=c("grunddata, "eksterne")

Thoughts, @lwjohnst86?

[lwjohnst86]

Yea, agree with the source. But I don't agree with including such specific projects in the function name like sdca and darter. These functions (and the package in general) should be generic enough so that other centers or research groups could use the package too.

[signekb]

Fair enough. How about:

• get_projectdb_path(center, project_id, source=c("grunddata", "eksterne"))
• get_project_path(center, project_id, source=c("grunddata", "eksterne"))
  • And then within this, if the project_id is darter's id, we'll handle that special case there.
• get_dbmanager_path(center, project_id, dbmanager_id, data=c("grunddata", "eksterne")

I'm not sure what other "centers" could be than "SDCA" and other Steno centers. Could it also be e.g., "AU", bc then we need another arg name. Maybe institute?

Thoughts, @lwjohnst86?

[lwjohnst86]

I think ProjectDB project databases from any academic institution (e.g. NCRR at AU has one). So center doesn't make sense. Could be name and id instead?

I also don't think _db_manager nor _DST_db_rawdata are paths DST created, I think that's something Marie/someone on the ProjectDB admin created. So we can't be certain every project database will have those paths.

[lwjohnst86]

We could have functions that try to guess potential paths from the name and id of the project database for Grunddata etc. I (hope/think) that every project database will have at least those two folders named that way, since I think that's something DST needs/looks for (since it is named that way in normal project folders).

[signekb]

Ah, so a function where you give it a project and it outputs paths with that id and e.g., Grunddata? And then another function for Eksterne data?

[lwjohnst86]

Yeaassss 👍 I also made #104 so that the user doesn't need to set Project ID all the time within a script.