Skip to content

Conversation

maya-the-mango
Copy link
Contributor

@maya-the-mango maya-the-mango commented Nov 19, 2024


labels: mergeable

Fixes: #issue

Motivation and Context

//: # Requesting bandits only when needed (when modelVersions differ) saves bandwidth! :)

Description

//: # Before requesting new bandits I do a simple comparison of loaded model version vs the newest model version contained in UFC

How has this been tested?

@maya-the-mango maya-the-mango marked this pull request as draft November 19, 2024 15:01
@maya-the-mango
Copy link
Contributor Author

2 unit tests are failing due to changes in test json file that happened before I even forked the repo. I've fixed them already though! But in this PR so we can ignore those here. :))

Copy link
Collaborator

@typotter typotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking great and definitely on the right track!
Just some comments I hope may make your testing easier.
Thanks for the lift in this SDK!

Copy link
Contributor

@aarsilv aarsilv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good! Only request is a more beefed up test for this!

Copy link
Contributor Author

@maya-the-mango maya-the-mango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I beefed up the tests! But I've never done this kind of fetch mocking shenanigans before in js, so I'm not 100% confident in the approach I took: passing mock response generator function to fetchSpy initializer and the updateUFC flag which serves as a semi global variable - I hate using this but nothing better comes to my mind tbh.

@maya-the-mango maya-the-mango marked this pull request as ready for review November 21, 2024 16:05
Copy link
Collaborator

@typotter typotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just about there!
Can't agree with you more about the awkwardness and repetition encountered when testing bandits.
Ides for how to refactor are most welcome!

Copy link
Contributor

@aarsilv aarsilv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating!

Approving, but before merging you'll want to do three things:

  1. Rebase on latest main and ensure package.json is just a patch (last number) bump
  2. Install this locally and put together a corresponding change and PR for the node SDK to make sure it works upstream as intended
  3. Get the sign off from @typotter who requested changes

expect(fetchSpy).toHaveBeenCalledTimes(3); // Once just for UFC, bandits should be skipped
});

const warmStartBanditReference = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these will be hoised to the top-level describe. Consider putting them in either in it's new nested describe() block along with the three tests that use it.

Comment on lines 367 to 371
* 1. initial call - 1 fetch for ufc 1 for bandits
* 2. 2nd call - 1 fetch for ufc only; bandits unchanged
* 3. 3rd call - new bandit ref injected to UFC; 2 fetches, because new bandit appeared
* 4. 4th call - we remove a bandit from ufc; 1 fetch because there is no need to update.
* The bandit removed from UFC should still be in memory.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the comment!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 agreed - super handy. It's easy to get lost when a test has to do multiple rounds of Arrange-Act-Assert to fully test functionality.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙌

Copy link
Collaborator

@typotter typotter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff. Going to see some solid improvement on our fastly request volumes with this.
LGTM, pending the version change and other comments from @aarsilv

@typotter typotter removed their assignment Nov 27, 2024
@maya-the-mango maya-the-mango force-pushed the selki/FF-3514-bandit-parameters-polling-only-when-needed branch from 490336d to 31cde96 Compare November 29, 2024 17:00
@maya-the-mango maya-the-mango force-pushed the selki/FF-3514-bandit-parameters-polling-only-when-needed branch from c8921c8 to baeb696 Compare December 16, 2024 08:15
@maya-the-mango maya-the-mango merged commit eeddcdc into main Dec 16, 2024
8 checks passed
@maya-the-mango maya-the-mango deleted the selki/FF-3514-bandit-parameters-polling-only-when-needed branch December 16, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants