Skip to content

TDL-20215: Add missing integration tests#42

Open
savan-chovatiya wants to merge 5 commits intocrest-masterfrom
TDL-20215-add-missing-integration-tests
Open

TDL-20215: Add missing integration tests#42
savan-chovatiya wants to merge 5 commits intocrest-masterfrom
TDL-20215-add-missing-integration-tests

Conversation

@savan-chovatiya
Copy link

Description of change

TDL-20215: Add missing integration tests

  • Added the following missing tests.
    • All fields test
    • Automatic fields test
    • Bookmarking test
    • Datatype test
  • Added missing assertion for discover mode test.

Manual QA steps

Risks

Rollback steps

  • revert this branch

expected_all_keys = stream_to_all_catalog_fields[stream]

# Tap is not writing _sdc_extra so removing it from expected keys.
expected_all_keys = expected_all_keys - {"_sdc_extra"}
Copy link
Contributor

@hpatel41 hpatel41 Sep 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can get _sdc_extra if we override generate_simple_csv_lines_typeA and send extra field data for one or two rows.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overrided generate_simple_csv_lines_typeA() function, and generated data for sdc_extra field.

def name(self):
return "tap_tester_sftp_automatic_fields"

def get_files(self):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems duplicating can we move this to base.py?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, Moved get_files() to base for the most used common structure, and override in other files if requires.

}
]

def setUp(self):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add function comments for each file's function.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added function comments

]

def setUp(self):
if not all([x for x in [os.getenv('TAP_SFTP_USERNAME'),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setup is also used in many files so move this to base.py and for diff lines, use super and add those lines.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SetUp() function is related to pytest so it will require in each file to set up something before running that particular test.

@dbshah1212 dbshah1212 self-requested a review September 20, 2022 10:48
Copy link

@dbshah1212 dbshah1212 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes looks good but build is failing may be need to update some stuffs at CCI side so can you please create a card for the same and do?

@hpatel41 hpatel41 requested a review from RushiT0122 October 11, 2022 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants