Skip to content

Commit b37fed9

Browse files
update Open Source Docs from Roblox internal teams
1 parent 6c8ae19 commit b37fed9

File tree

3 files changed

+18
-11
lines changed

3 files changed

+18
-11
lines changed
Lines changed: 2 additions & 2 deletions
Loading

content/en-us/production/experiments.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,11 @@ Players.PlayerAdded:Connect(onPlayerAdded)
141141

142142
- You must call `Class.ConfigService:GetConfigForPlayerAsync()|GetConfigForPlayerAsync()` separately for each player; `Class.ConfigService:GetConfigAsync()|GetConfigAsync()` does not apply experiments.
143143
- After you call `Class.ConfigSnapshot:GetValue()|GetValue()` on a player-specific snapshot, the player associated with the snapshot is enrolled in the experiment for that key and that key only. All subsequent calls to the method return the same control or variant for the duration of the experiment. Only the first call is random.
144+
145+
<Alert severity="success">
146+
Wait to call `Class.ConfigSnapshot:GetValue()|GetValue()` until you need it. Calling `Class.ConfigSnapshot:GetValue()|GetValue()` too early can cause you to enroll players who never interact with the part of the experience you're experimenting on.
147+
</Alert>
148+
144149
- Enrollment in experiments isn't limited to new users. Even if a user previously received a value from `Class.ConfigService:GetConfigAsync()|GetConfigAsync()`, you can still enroll them in an experiement using a player-specific snapshot from `Class.ConfigService:GetConfigForPlayerAsync()|GetConfigForPlayerAsync()`.
145150
- If a key in a player-specific snapshot doesn't have an active experiment, `Class.ConfigSnapshot:GetValue()|GetValue()` returns the standard config value (or nil if it has no value).
146151

@@ -207,10 +212,12 @@ For convenience, the results page lets you replace the default config value with
207212

208213
- **Don't act without statistical significance.** Even seemingly large changes in player behavior might not be statistically significant, generally due to small sample size. If a change isn't statistically significant, ignore it.
209214

210-
- **Avoid changes during experiments**. Major bugs of course need fixes, but changes to experience content can impact player behavior and invalidate your results, even if the changes **seem** unrelated to your experiment.
215+
- **Avoid changes during experiments**. Major bugs of course need fixes, but changes to experience content can impact player behavior and invalidate your results, even if the changes **seem** unrelated to your experiment. Similarly, only run experiments simultaneously if you're confident they won't interact with each other.
211216

212217
- **Use confidence intervals for deep dives** into metrics and to check for borderline cases of statistical significance. If the confidence interval is too wide, the metric might never reach statistical significance.
213218

214219
- If one metric is significantly up and another significantly down, you have to **decide whether the trade-off is worth it**, possibly in conjunction with other statistically significant movements.
215220

216221
- Experiments provide strong signal, but **statistical significance deals in probabilities, not certainties**—hence the confidence interval. Data variability, sample size, and magnitude of the change all impact the probability of detecting whether a variant affected player behavior. Any action you take based on the results of an experiment should be balanced against qualitative data like player feedback and your overall vision for the experience.
222+
223+
- **Document your findings and decisions.** Even if you don't use them to run additional experiments, having a body of knowledge and evidence can inform how you design your experiences.

content/en-us/reference/engine/datatypes/BrickColor.yaml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1604,29 +1604,29 @@ constructors:
16041604
tags: []
16051605
deprecation_message: ''
16061606
- name: BrickColor.Red
1607-
summary: Returns the "Bright Red" `Datatype.BrickColor`.
1608-
description: Returns the "Bright Red" `Datatype.BrickColor`.
1607+
summary: Returns the "Bright red" `Datatype.BrickColor`.
1608+
description: Returns the "Bright red" `Datatype.BrickColor`.
16091609
parameters: []
16101610
code_samples: []
16111611
tags: []
16121612
deprecation_message: ''
16131613
- name: BrickColor.Yellow
1614-
summary: Returns the "Bright Yellow" `Datatype.BrickColor`.
1615-
description: Returns the "Bright Yellow" `Datatype.BrickColor`.
1614+
summary: Returns the "Bright yellow" `Datatype.BrickColor`.
1615+
description: Returns the "Bright yellow" `Datatype.BrickColor`.
16161616
parameters: []
16171617
code_samples: []
16181618
tags: []
16191619
deprecation_message: ''
16201620
- name: BrickColor.Green
1621-
summary: Returns the "Dark Green" `Datatype.BrickColor`.
1622-
description: Returns the "Dark Green" `Datatype.BrickColor`.
1621+
summary: Returns the "Dark green" `Datatype.BrickColor`.
1622+
description: Returns the "Dark green" `Datatype.BrickColor`.
16231623
parameters: []
16241624
code_samples: []
16251625
tags: []
16261626
deprecation_message: ''
16271627
- name: BrickColor.Blue
1628-
summary: Returns the "Bright Blue" `Datatype.BrickColor`.
1629-
description: Returns the "Bright Blue" `Datatype.BrickColor`.
1628+
summary: Returns the "Bright blue" `Datatype.BrickColor`.
1629+
description: Returns the "Bright blue" `Datatype.BrickColor`.
16301630
parameters: []
16311631
code_samples: []
16321632
tags: []

0 commit comments

Comments
 (0)