Skip to content

Commit fec53d2

Browse files
feat: Dh-20517: Add ConcurrencyControl and Selectable wrappers in Py Server API (#7295)
Co-authored-by: margaretkennedy <[email protected]>
1 parent 16cdded commit fec53d2

File tree

13 files changed

+449
-24
lines changed

13 files changed

+449
-24
lines changed

docs/python/conceptual/query-engine/parallelization.md

Lines changed: 112 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,117 @@ other table — the original `my_table`. Since they are independent of each othe
6969
modified rows it is possible for the query engine to process the new rows into `my_table_updated`, `my_table_filtered1`
7070
and `my_table_filtered2` at the same time. However, since `merged_tables` depends on those three tables, the query
7171
engine cannot update the result of the [`merge`](../../reference/table-operations/merge/merge.md) operation until after
72-
the `update()` and `where()`s for those three tables have been processed.
72+
the [`update`](../../reference/table-operations/select/update.md) and [`where`](../../reference/table-operations/filter/where.md)s for those three tables have been processed.
73+
74+
### Controlling Concurrency for `select`, `update` and `where`
75+
76+
The [`select`](../../reference/table-operations/select/select.md), [`update`](../../reference/table-operations/select/update.md), and [`where`](../../reference/table-operations/filter/where.md) operations can parallelize within a single where clause or column expression. This can greatly improve throughput by using multiple threads to read existing columns or compute functions.
77+
78+
Deephaven can only parallelize an expression if it is _stateless_, meaning it does not depend on any mutable external inputs or the order in which rows are evaluated. Many operations, such as string manipulation or arithmetic on one or more input columns, are stateless.
79+
80+
By default, the Deephaven engine assumes that expressions are stateful (not stateless). For [`select`](../../reference/table-operations/select/select.md) and [`update`](../../reference/table-operations/select/update.md), you can change the configuration property `QueryTable.statelessSelectByDefault` to `true` to make columns stateless by default. For filters, change the property `QueryTable.statelessFiltersByDefault`.
81+
82+
> [!NOTE]
83+
> In a future version of Deephaven, filters and selectables will be stateless by default.
84+
85+
The [`ConcurrencyControl`](https://docs.deephaven.io/core/pydoc/code/concurrency_control.html#deephaven.concurrency_control.ConcurrencyControl) interface allows you to control the behavior of [`Filter`](https://docs.deephaven.io/core/pydoc/code/deephaven.filters.html) (where clause) and [`Selectable`](https://docs.deephaven.io/core/pydoc/code/deephaven.table.html#deephaven.table.Selectable) objects (update and select table operations).
86+
87+
To explicitly mark a Selectable or Filter as stateful, use the `with_serial` method.
88+
89+
- A serial Filter cannot be reordered with respect to other Filters. Every input row to a serial Filter is evaluated in order.
90+
- When a Selectable is serial, every row for that column is evaluated in order.
91+
- For Selectables, additional ordering constraints are controlled by `QueryTable.SERIAL_SELECT_IMPLICIT_BARRIERS`, which is set by the property `QueryTable.serialSelectImplicitBarriers`. The default value is the inverse of `QueryTable.statelessSelectByDefault`:
92+
- When Selectables are stateless by default, no implicit barriers are added (`QueryTable.SERIAL_SELECT_IMPLICIT_BARRIERS` is false).
93+
- When Selectables are stateful by default, implicit barriers are added (`QueryTable.SERIAL_SELECT_IMPLICIT_BARRIERS` is true).
94+
- If `QueryTable.SERIAL_SELECT_IMPLICIT_BARRIERS` is false, no additional ordering between expressions is imposed. As with every [`select`](../../reference/table-operations/select/select.md) or [`update`](../../reference/table-operations/select/update.md) call, if column B references column A, then column A is evaluated before column B. To impose further ordering constraints, use barriers.
95+
- If `QueryTable.SERIAL_SELECT_IMPLICIT_BARRIERS` is true, a serial Selectable acts as an absolute barrier with respect to all other serial Selectables. This prohibits serial Selectables from being evaluated concurrently, permitting them to access global state. Non-serial Selectables may be reordered with respect to a serial Selectable.
96+
97+
Filters and Selectables may declare a [`Barrier`](https://docs.deephaven.io/core/pydoc/code/deephaven.concurrency_control.html#deephaven.concurrency_control.Barrier). A barrier is an opaque object (compared using reference equality) used to control evaluation order between Filters or Selectables.
98+
99+
Subsequent Filters or Selectables may respect a previously declared barrier:
100+
101+
- If a Filter respects a barrier, it cannot begin evaluation until the Filter that declared the barrier has been completely evaluated.
102+
- If a Selectable respects a barrier, it cannot begin evaluation until the Selectable that declared the barrier has been completely evaluated.
103+
104+
In this code block, two columns call a Python stateful function that is not thread-safe:
105+
106+
```python order=null
107+
from deephaven import empty_table
108+
109+
counter = 0
110+
111+
112+
def get_and_increment_counter() -> int:
113+
global counter
114+
ret = counter
115+
counter += 1
116+
return ret
117+
118+
119+
t = empty_table(1_000_000).update(
120+
["A = get_and_increment_counter()", "B = get_and_increment_counter()"]
121+
)
122+
```
123+
124+
Deephaven's default behavior is to treat both `A` and `B` statefully, therefore the table is equivalent to:
125+
126+
```python order=null
127+
from deephaven import empty_table
128+
129+
t = empty_table(1_000_000).update(["A=i", "B=1_000_000 + i"])
130+
```
131+
132+
However, if the columns were marked as stateless (e.g., if `QueryTable.statelessSelectByDefault` were `true`), the rows from either column could be evaluated in any order, potentially causing race conditions. To ensure that all rows of `A` are evaluated before any rows of `B` begin evaluation, use a barrier:
133+
134+
```python order=null
135+
from deephaven.concurrency_control import Barrier
136+
from deephaven.table import Selectable
137+
from deephaven import empty_table
138+
139+
counter = 0
140+
141+
142+
def get_and_increment_counter() -> int:
143+
global counter
144+
ret = counter
145+
counter += 1
146+
return ret
147+
148+
149+
barrier = Barrier()
150+
col_a = Selectable.parse(
151+
formula="A = get_and_increment_counter()"
152+
).with_declared_barriers(barrier)
153+
col_b = Selectable.parse(
154+
formula="B = get_and_increment_counter()"
155+
).with_respected_barriers(barrier)
156+
157+
t = empty_table(1_000_000).update([col_a, col_b])
158+
```
159+
160+
Alternatively, you can ensure that values of `A` are evaluated in order by using `with_serial` on a Selectable:
161+
162+
```python order=null
163+
from deephaven.concurrency_control import Barrier
164+
from deephaven.table import Selectable
165+
from deephaven import empty_table
166+
167+
counter = 0
168+
169+
170+
def get_and_increment_counter() -> int:
171+
global counter
172+
ret = counter
173+
counter += 1
174+
return ret
175+
176+
177+
barrier = Barrier()
178+
col_a = Selectable.parse(formula="A = get_and_increment_counter()").with_serial()
179+
col_b = Selectable.parse(formula="B = get_and_increment_counter()")
180+
181+
t = empty_table(1_000_000).update([col_a, col_b])
182+
```
73183

74184
### Managing thread pool sizes
75185

@@ -83,8 +193,7 @@ described in the table below:
83193
| PeriodicUpdateGraph.updateThreads | -1 | Determines the number of threads available for parallel processing of the Update Graph Processor refresh cycle. |
84194

85195
Setting either of these properties to `-1` instructs Deephaven to use all available processors. The number of available
86-
processors is retrieved from the Java Virtual Machine at Deephaven startup,
87-
using [Runtime.availableProcessors()](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Runtime.html#availableProcessors()).
196+
processors is retrieved from the Java Virtual Machine at Deephaven startup, using [Runtime.availableProcessors()](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Runtime.html#availableProcessors()).
88197

89198
### Related documentation
90199

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"file":"conceptual/query-engine/parallelization.md","objects":{"t":{"type":"Table","data":{"columns":[{"name":"A","type":"int"},{"name":"B","type":"int"}],"rows":[[{"value":"0"},{"value":"1,000,000"}],[{"value":"1"},{"value":"1,000,001"}],[{"value":"2"},{"value":"1,000,002"}],[{"value":"3"},{"value":"1,000,003"}],[{"value":"4"},{"value":"1,000,004"}],[{"value":"5"},{"value":"1,000,005"}],[{"value":"6"},{"value":"1,000,006"}],[{"value":"7"},{"value":"1,000,007"}],[{"value":"8"},{"value":"1,000,008"}],[{"value":"9"},{"value":"1,000,009"}],[{"value":"10"},{"value":"1,000,010"}],[{"value":"11"},{"value":"1,000,011"}],[{"value":"12"},{"value":"1,000,012"}],[{"value":"13"},{"value":"1,000,013"}],[{"value":"14"},{"value":"1,000,014"}],[{"value":"15"},{"value":"1,000,015"}],[{"value":"16"},{"value":"1,000,016"}],[{"value":"17"},{"value":"1,000,017"}],[{"value":"18"},{"value":"1,000,018"}],[{"value":"19"},{"value":"1,000,019"}],[{"value":"20"},{"value":"1,000,020"}],[{"value":"21"},{"value":"1,000,021"}],[{"value":"22"},{"value":"1,000,022"}],[{"value":"23"},{"value":"1,000,023"}],[{"value":"24"},{"value":"1,000,024"}],[{"value":"25"},{"value":"1,000,025"}],[{"value":"26"},{"value":"1,000,026"}],[{"value":"27"},{"value":"1,000,027"}],[{"value":"28"},{"value":"1,000,028"}],[{"value":"29"},{"value":"1,000,029"}],[{"value":"30"},{"value":"1,000,030"}],[{"value":"31"},{"value":"1,000,031"}],[{"value":"32"},{"value":"1,000,032"}],[{"value":"33"},{"value":"1,000,033"}],[{"value":"34"},{"value":"1,000,034"}],[{"value":"35"},{"value":"1,000,035"}],[{"value":"36"},{"value":"1,000,036"}],[{"value":"37"},{"value":"1,000,037"}],[{"value":"38"},{"value":"1,000,038"}],[{"value":"39"},{"value":"1,000,039"}],[{"value":"40"},{"value":"1,000,040"}],[{"value":"41"},{"value":"1,000,041"}],[{"value":"42"},{"value":"1,000,042"}],[{"value":"43"},{"value":"1,000,043"}],[{"value":"44"},{"value":"1,000,044"}],[{"value":"45"},{"value":"1,000,045"}],[{"value":"46"},{"value":"1,000,046"}],[{"value":"47"},{"value":"1,000,047"}],[{"value":"48"},{"value":"1,000,048"}],[{"value":"49"},{"value":"1,000,049"}],[{"value":"50"},{"value":"1,000,050"}],[{"value":"51"},{"value":"1,000,051"}],[{"value":"52"},{"value":"1,000,052"}],[{"value":"53"},{"value":"1,000,053"}],[{"value":"54"},{"value":"1,000,054"}],[{"value":"55"},{"value":"1,000,055"}],[{"value":"56"},{"value":"1,000,056"}],[{"value":"57"},{"value":"1,000,057"}],[{"value":"58"},{"value":"1,000,058"}],[{"value":"59"},{"value":"1,000,059"}],[{"value":"60"},{"value":"1,000,060"}],[{"value":"61"},{"value":"1,000,061"}],[{"value":"62"},{"value":"1,000,062"}],[{"value":"63"},{"value":"1,000,063"}],[{"value":"64"},{"value":"1,000,064"}],[{"value":"65"},{"value":"1,000,065"}],[{"value":"66"},{"value":"1,000,066"}],[{"value":"67"},{"value":"1,000,067"}],[{"value":"68"},{"value":"1,000,068"}],[{"value":"69"},{"value":"1,000,069"}],[{"value":"70"},{"value":"1,000,070"}],[{"value":"71"},{"value":"1,000,071"}],[{"value":"72"},{"value":"1,000,072"}],[{"value":"73"},{"value":"1,000,073"}],[{"value":"74"},{"value":"1,000,074"}],[{"value":"75"},{"value":"1,000,075"}],[{"value":"76"},{"value":"1,000,076"}],[{"value":"77"},{"value":"1,000,077"}],[{"value":"78"},{"value":"1,000,078"}],[{"value":"79"},{"value":"1,000,079"}],[{"value":"80"},{"value":"1,000,080"}],[{"value":"81"},{"value":"1,000,081"}],[{"value":"82"},{"value":"1,000,082"}],[{"value":"83"},{"value":"1,000,083"}],[{"value":"84"},{"value":"1,000,084"}],[{"value":"85"},{"value":"1,000,085"}],[{"value":"86"},{"value":"1,000,086"}],[{"value":"87"},{"value":"1,000,087"}],[{"value":"88"},{"value":"1,000,088"}],[{"value":"89"},{"value":"1,000,089"}],[{"value":"90"},{"value":"1,000,090"}],[{"value":"91"},{"value":"1,000,091"}],[{"value":"92"},{"value":"1,000,092"}],[{"value":"93"},{"value":"1,000,093"}],[{"value":"94"},{"value":"1,000,094"}],[{"value":"95"},{"value":"1,000,095"}],[{"value":"96"},{"value":"1,000,096"}],[{"value":"97"},{"value":"1,000,097"}],[{"value":"98"},{"value":"1,000,098"}],[{"value":"99"},{"value":"1,000,099"}]]}}}}
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"file":"conceptual/query-engine/parallelization.md","objects":{"t":{"type":"Table","data":{"columns":[{"name":"A","type":"long"},{"name":"B","type":"long"}],"rows":[[{"value":"0"},{"value":"1,000,000"}],[{"value":"1"},{"value":"1,000,001"}],[{"value":"2"},{"value":"1,000,002"}],[{"value":"3"},{"value":"1,000,003"}],[{"value":"4"},{"value":"1,000,004"}],[{"value":"5"},{"value":"1,000,005"}],[{"value":"6"},{"value":"1,000,006"}],[{"value":"7"},{"value":"1,000,007"}],[{"value":"8"},{"value":"1,000,008"}],[{"value":"9"},{"value":"1,000,009"}],[{"value":"10"},{"value":"1,000,010"}],[{"value":"11"},{"value":"1,000,011"}],[{"value":"12"},{"value":"1,000,012"}],[{"value":"13"},{"value":"1,000,013"}],[{"value":"14"},{"value":"1,000,014"}],[{"value":"15"},{"value":"1,000,015"}],[{"value":"16"},{"value":"1,000,016"}],[{"value":"17"},{"value":"1,000,017"}],[{"value":"18"},{"value":"1,000,018"}],[{"value":"19"},{"value":"1,000,019"}],[{"value":"20"},{"value":"1,000,020"}],[{"value":"21"},{"value":"1,000,021"}],[{"value":"22"},{"value":"1,000,022"}],[{"value":"23"},{"value":"1,000,023"}],[{"value":"24"},{"value":"1,000,024"}],[{"value":"25"},{"value":"1,000,025"}],[{"value":"26"},{"value":"1,000,026"}],[{"value":"27"},{"value":"1,000,027"}],[{"value":"28"},{"value":"1,000,028"}],[{"value":"29"},{"value":"1,000,029"}],[{"value":"30"},{"value":"1,000,030"}],[{"value":"31"},{"value":"1,000,031"}],[{"value":"32"},{"value":"1,000,032"}],[{"value":"33"},{"value":"1,000,033"}],[{"value":"34"},{"value":"1,000,034"}],[{"value":"35"},{"value":"1,000,035"}],[{"value":"36"},{"value":"1,000,036"}],[{"value":"37"},{"value":"1,000,037"}],[{"value":"38"},{"value":"1,000,038"}],[{"value":"39"},{"value":"1,000,039"}],[{"value":"40"},{"value":"1,000,040"}],[{"value":"41"},{"value":"1,000,041"}],[{"value":"42"},{"value":"1,000,042"}],[{"value":"43"},{"value":"1,000,043"}],[{"value":"44"},{"value":"1,000,044"}],[{"value":"45"},{"value":"1,000,045"}],[{"value":"46"},{"value":"1,000,046"}],[{"value":"47"},{"value":"1,000,047"}],[{"value":"48"},{"value":"1,000,048"}],[{"value":"49"},{"value":"1,000,049"}],[{"value":"50"},{"value":"1,000,050"}],[{"value":"51"},{"value":"1,000,051"}],[{"value":"52"},{"value":"1,000,052"}],[{"value":"53"},{"value":"1,000,053"}],[{"value":"54"},{"value":"1,000,054"}],[{"value":"55"},{"value":"1,000,055"}],[{"value":"56"},{"value":"1,000,056"}],[{"value":"57"},{"value":"1,000,057"}],[{"value":"58"},{"value":"1,000,058"}],[{"value":"59"},{"value":"1,000,059"}],[{"value":"60"},{"value":"1,000,060"}],[{"value":"61"},{"value":"1,000,061"}],[{"value":"62"},{"value":"1,000,062"}],[{"value":"63"},{"value":"1,000,063"}],[{"value":"64"},{"value":"1,000,064"}],[{"value":"65"},{"value":"1,000,065"}],[{"value":"66"},{"value":"1,000,066"}],[{"value":"67"},{"value":"1,000,067"}],[{"value":"68"},{"value":"1,000,068"}],[{"value":"69"},{"value":"1,000,069"}],[{"value":"70"},{"value":"1,000,070"}],[{"value":"71"},{"value":"1,000,071"}],[{"value":"72"},{"value":"1,000,072"}],[{"value":"73"},{"value":"1,000,073"}],[{"value":"74"},{"value":"1,000,074"}],[{"value":"75"},{"value":"1,000,075"}],[{"value":"76"},{"value":"1,000,076"}],[{"value":"77"},{"value":"1,000,077"}],[{"value":"78"},{"value":"1,000,078"}],[{"value":"79"},{"value":"1,000,079"}],[{"value":"80"},{"value":"1,000,080"}],[{"value":"81"},{"value":"1,000,081"}],[{"value":"82"},{"value":"1,000,082"}],[{"value":"83"},{"value":"1,000,083"}],[{"value":"84"},{"value":"1,000,084"}],[{"value":"85"},{"value":"1,000,085"}],[{"value":"86"},{"value":"1,000,086"}],[{"value":"87"},{"value":"1,000,087"}],[{"value":"88"},{"value":"1,000,088"}],[{"value":"89"},{"value":"1,000,089"}],[{"value":"90"},{"value":"1,000,090"}],[{"value":"91"},{"value":"1,000,091"}],[{"value":"92"},{"value":"1,000,092"}],[{"value":"93"},{"value":"1,000,093"}],[{"value":"94"},{"value":"1,000,094"}],[{"value":"95"},{"value":"1,000,095"}],[{"value":"96"},{"value":"1,000,096"}],[{"value":"97"},{"value":"1,000,097"}],[{"value":"98"},{"value":"1,000,098"}],[{"value":"99"},{"value":"1,000,099"}]]}}}}
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"file":"conceptual/query-engine/parallelization.md","objects":{"t":{"type":"Table","data":{"columns":[{"name":"A","type":"long"},{"name":"B","type":"long"}],"rows":[[{"value":"0"},{"value":"1,000,000"}],[{"value":"1"},{"value":"1,000,001"}],[{"value":"2"},{"value":"1,000,002"}],[{"value":"3"},{"value":"1,000,003"}],[{"value":"4"},{"value":"1,000,004"}],[{"value":"5"},{"value":"1,000,005"}],[{"value":"6"},{"value":"1,000,006"}],[{"value":"7"},{"value":"1,000,007"}],[{"value":"8"},{"value":"1,000,008"}],[{"value":"9"},{"value":"1,000,009"}],[{"value":"10"},{"value":"1,000,010"}],[{"value":"11"},{"value":"1,000,011"}],[{"value":"12"},{"value":"1,000,012"}],[{"value":"13"},{"value":"1,000,013"}],[{"value":"14"},{"value":"1,000,014"}],[{"value":"15"},{"value":"1,000,015"}],[{"value":"16"},{"value":"1,000,016"}],[{"value":"17"},{"value":"1,000,017"}],[{"value":"18"},{"value":"1,000,018"}],[{"value":"19"},{"value":"1,000,019"}],[{"value":"20"},{"value":"1,000,020"}],[{"value":"21"},{"value":"1,000,021"}],[{"value":"22"},{"value":"1,000,022"}],[{"value":"23"},{"value":"1,000,023"}],[{"value":"24"},{"value":"1,000,024"}],[{"value":"25"},{"value":"1,000,025"}],[{"value":"26"},{"value":"1,000,026"}],[{"value":"27"},{"value":"1,000,027"}],[{"value":"28"},{"value":"1,000,028"}],[{"value":"29"},{"value":"1,000,029"}],[{"value":"30"},{"value":"1,000,030"}],[{"value":"31"},{"value":"1,000,031"}],[{"value":"32"},{"value":"1,000,032"}],[{"value":"33"},{"value":"1,000,033"}],[{"value":"34"},{"value":"1,000,034"}],[{"value":"35"},{"value":"1,000,035"}],[{"value":"36"},{"value":"1,000,036"}],[{"value":"37"},{"value":"1,000,037"}],[{"value":"38"},{"value":"1,000,038"}],[{"value":"39"},{"value":"1,000,039"}],[{"value":"40"},{"value":"1,000,040"}],[{"value":"41"},{"value":"1,000,041"}],[{"value":"42"},{"value":"1,000,042"}],[{"value":"43"},{"value":"1,000,043"}],[{"value":"44"},{"value":"1,000,044"}],[{"value":"45"},{"value":"1,000,045"}],[{"value":"46"},{"value":"1,000,046"}],[{"value":"47"},{"value":"1,000,047"}],[{"value":"48"},{"value":"1,000,048"}],[{"value":"49"},{"value":"1,000,049"}],[{"value":"50"},{"value":"1,000,050"}],[{"value":"51"},{"value":"1,000,051"}],[{"value":"52"},{"value":"1,000,052"}],[{"value":"53"},{"value":"1,000,053"}],[{"value":"54"},{"value":"1,000,054"}],[{"value":"55"},{"value":"1,000,055"}],[{"value":"56"},{"value":"1,000,056"}],[{"value":"57"},{"value":"1,000,057"}],[{"value":"58"},{"value":"1,000,058"}],[{"value":"59"},{"value":"1,000,059"}],[{"value":"60"},{"value":"1,000,060"}],[{"value":"61"},{"value":"1,000,061"}],[{"value":"62"},{"value":"1,000,062"}],[{"value":"63"},{"value":"1,000,063"}],[{"value":"64"},{"value":"1,000,064"}],[{"value":"65"},{"value":"1,000,065"}],[{"value":"66"},{"value":"1,000,066"}],[{"value":"67"},{"value":"1,000,067"}],[{"value":"68"},{"value":"1,000,068"}],[{"value":"69"},{"value":"1,000,069"}],[{"value":"70"},{"value":"1,000,070"}],[{"value":"71"},{"value":"1,000,071"}],[{"value":"72"},{"value":"1,000,072"}],[{"value":"73"},{"value":"1,000,073"}],[{"value":"74"},{"value":"1,000,074"}],[{"value":"75"},{"value":"1,000,075"}],[{"value":"76"},{"value":"1,000,076"}],[{"value":"77"},{"value":"1,000,077"}],[{"value":"78"},{"value":"1,000,078"}],[{"value":"79"},{"value":"1,000,079"}],[{"value":"80"},{"value":"1,000,080"}],[{"value":"81"},{"value":"1,000,081"}],[{"value":"82"},{"value":"1,000,082"}],[{"value":"83"},{"value":"1,000,083"}],[{"value":"84"},{"value":"1,000,084"}],[{"value":"85"},{"value":"1,000,085"}],[{"value":"86"},{"value":"1,000,086"}],[{"value":"87"},{"value":"1,000,087"}],[{"value":"88"},{"value":"1,000,088"}],[{"value":"89"},{"value":"1,000,089"}],[{"value":"90"},{"value":"1,000,090"}],[{"value":"91"},{"value":"1,000,091"}],[{"value":"92"},{"value":"1,000,092"}],[{"value":"93"},{"value":"1,000,093"}],[{"value":"94"},{"value":"1,000,094"}],[{"value":"95"},{"value":"1,000,095"}],[{"value":"96"},{"value":"1,000,096"}],[{"value":"97"},{"value":"1,000,097"}],[{"value":"98"},{"value":"1,000,098"}],[{"value":"99"},{"value":"1,000,099"}]]}}}}

0 commit comments

Comments
 (0)