Skip to content

Skip examples or tests that need credentials if not available#484

Open
PietrH wants to merge 26 commits intomainfrom
pass-win-devel
Open

Skip examples or tests that need credentials if not available#484
PietrH wants to merge 26 commits intomainfrom
pass-win-devel

Conversation

@PietrH
Copy link
Member

@PietrH PietrH commented Feb 12, 2026

Some examples or tests need credentials to be stored in order to run. This is no big deal but will break CRAN submission or using devtools::check_win_release(). This PR gets this out of the way.

I add two helpers, one to check if credentials are set, this is used for conditional examples and for the other function: a testthat skip helper that skips a test if no credentials are set.

It should be easy to adapt these helpers to keyring later on. They may need no modification whatsoever.

Possible followup: Some tests are already skipped if there is no local database connection, in fact, these should also be skipped if there are no credentials set. I suppose I could repeat the skip or call it from within skip_if_not_localdb().


This is a nice to have, and not a blocker for the ETN API release.

I think I was done working on this, I can't see why I didn't open this for review earlier.

@PietrH PietrH self-assigned this Feb 12, 2026
@PietrH PietrH changed the base branch from main to opencpu February 12, 2026 14:34
@PietrH
Copy link
Member Author

PietrH commented Feb 12, 2026

There is no issue for this PR

@PietrH
Copy link
Member Author

PietrH commented Feb 12, 2026

The package is also too big to be sent via devtools::check_win_release(), i removed the fixtures, built the package locally and uploaded via the webform; I'm not sure if it worked.

@codecov
Copy link

codecov bot commented Feb 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.74%. Comparing base (d9bf468) to head (0b5a88c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #484      +/-   ##
==========================================
+ Coverage   85.65%   86.74%   +1.09%     
==========================================
  Files          26       26              
  Lines         746      747       +1     
==========================================
+ Hits          639      648       +9     
+ Misses        107       99       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@PietrH
Copy link
Member Author

PietrH commented Feb 12, 2026

This seems to have a very negative effect on package coverage, does covr somehow skip these tests as well? It should have access to credentials since it worked just fine before!

@PietrH
Copy link
Member Author

PietrH commented Feb 12, 2026

CI also has much more skips than expected, something is wrong with my helper and covr caught it!

@PietrH PietrH changed the title [WIP] Skip examples or tests that need credentials if not available Skip examples or tests that need credentials if not available Feb 24, 2026
@PietrH PietrH marked this pull request as ready for review February 24, 2026 08:36
@PietrH PietrH requested a review from peterdesmet February 24, 2026 09:22
Copy link
Member

@peterdesmet peterdesmet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor suggestions

}
\examples{
\dontshow{if (etn:::credentials_are_set()) withAutoprint(\{ # examplesIf}
# example code
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the literal comment # example code? If yes, then I would remove.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also curious how

dontshow{if (etn:::credentials_are_set()) withAutoprint(\{ # examplesIf}

Renders on the pkgdown website.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Member Author

@PietrH PietrH Feb 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the system that renders the pkgdown website has access to credentials (CI, or dev machine) it'll just run the examples. CRAN for example, will not.

Note the progress is still reported even though the function is not called interactively, maybe I should stop that from happening.

@@ -1,3 +1,5 @@
skip_if_no_authentication()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be done outside a test_that() function?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for other occurrences where the function is called at the beginning of the test file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but this has the advantage of skipping all tests of the test-* file without me having to repeat it for every test. But it has the disadvantage of not counting the number of tests that are skipped.

I took the easiest route most of the time, do you prefer me switching to skipping in the test_that({}) context every time?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it's probably better to skip within the test (best practice).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I'll switch that around and let you know.

@PietrH PietrH changed the base branch from opencpu to main February 24, 2026 13:27
@PietrH
Copy link
Member Author

PietrH commented Feb 24, 2026

Tested on the rstudio server to ensure it also skips when it's supposed to skip when there is db access. Made a few changes in the hopes to fix #494

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants