[DEV-12772] - Treasury Account Spark Downloads

[zachflanders-frb]

Description:
This PR adds Treasury Account Downloads to the generate spark downloads command.

Technical details:
The treasury account downloads were able to use the account download delta table with the addition of a few columns specific to the treasury download.

Requirements for PR merge:

1. Unit & integration tests updated
2. API documentation updated
3. Necessary PR reviewers:
   • Backend
4. Jira Ticket DEV-12772:
   • Link to this Pull-Request

[zachflanders-frb]

Black and Flake8 disagree on this. I vote we override flake8.

[zachflanders-frb]

More detail here: psf/black#4173

[sethstoudenmier]

Approving because the changes look good to me, but I do have a conceptual question about structure as build this out more.

[sethstoudenmier]

Only question I have is around file structure and maybe you have already thought about this. The File A and B downloads will be different columns than the File C (Award Financial). This means that ideally, they could re-use the same dataframe filter logic, but the columns will differ. This gets me to my actual question:

Should we have this structured such that the Abstract classes are broken apart from the specific download implementations? For example, right now you have the Federal and Treasury account implementations in this file for the File C download. But if File A and B are going to need different implementations, would we want to group them into this same file?

[zachflanders-frb]

The organization of these classes feels like it could evolve. It seems like the overall decision tree looks something like

graph LR;
A((Account Level)) --> B[Federal Account];
A((Account Level)) --> C[Treasury Account];
B[Federal Account] --> D((Submission Type));
C[Treasury Account] --> E((Submission Type));
D((Submission Type)) --> F[Account Download];
D((Submission Type)) --> G[Object Class];
D((Submission Type)) --> H[Award Financial];
E((Submission Type)) --> I[Account Download];
E((Submission Type)) --> J[Object Class];
E((Submission Type)) --> K[Award Financial];

Loading

In some of my WIP on File A/B downloads I have this on the abstract class:

    @property
    @abstractmethod
    def account_balances(self) -> DataFrame: ...
    
    @property
    @abstractmethod
    def object_class_program_activity(self) -> DataFrame: ...

    @property
    @abstractmethod
    def award_financial(self) -> DataFrame: ...

    @property
    def source_dfs(self) -> list[DataFrame]:
        return [getattr(self, submission_type) for submission_type in self.submission_types]

Many of the filtering methods and other useful functions can be reused across the different submission types.