[DEV-12587] - File A (Account Balances) spark downloads #4441
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
This PR add the
appropriation_account_balancestable to theglobal_tempspark database. It enables download of the account balances (File A) for both Treasury Accounts and Federal Accounts.Technical Details:
The
appropriation_account_balancestable is just over 500,000 rows, so it seems reasonable to load it into the global_temp instead of using thecreate_delta_tableandload_query_to_deltacommands to create a delta table. The joins to submission attributes and other tables are includes in the pyspark dataframe api format.Requirements for PR Merge: