Skip to content

Commit 3474df9

Browse files
shortcutsshortcuts
committed
fix: some
1 parent 1989e1f commit 3474df9

File tree

7 files changed

+74
-58
lines changed

7 files changed

+74
-58
lines changed

‎clients/algoliasearch-client-java/.java-version‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
21.0.8
1+
21.0.6

‎clients/algoliasearch-client-php/lib/FormDataProcessor.php‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@
1212
*/
1313

1414
/**
15-
* Search API.
15+
* Ingestion API.
1616
*
17-
* The Algolia Search API lets you search, configure, and manage your indices and records. ## Client libraries Use Algolia's API clients and libraries to reliably integrate Algolia's APIs with your apps. The official API clients are covered by Algolia's [Service Level Agreement](https://www.algolia.com/policies/sla/). See: [Algolia's ecosystem](https://www.algolia.com/doc/guides/getting-started/how-algolia-works/in-depth/ecosystem/) ## Base URLs The base URLs for requests to the Search API are: - `https://{APPLICATION_ID}.algolia.net` - `https://{APPLICATION_ID}-dsn.algolia.net`. If your subscription includes a [Distributed Search Network](https://dashboard.algolia.com/infra), this ensures that requests are sent to servers closest to users. Both URLs provide high availability by distributing requests with load balancing. **All requests must use HTTPS.** ## Retry strategy To guarantee high availability, implement a retry strategy for all API requests using the URLs of your servers as fallbacks: - `https://{APPLICATION_ID}-1.algolianet.com` - `https://{APPLICATION_ID}-2.algolianet.com` - `https://{APPLICATION_ID}-3.algolianet.com` These URLs use a different DNS provider than the primary URLs. You should randomize this list to ensure an even load across the three servers. All Algolia API clients implement this retry strategy. ## Authentication To authenticate your API requests, add these headers: - `x-algolia-application-id`. Your Algolia application ID. - `x-algolia-api-key`. An API key with the necessary permissions to make the request. The required access control list (ACL) to make a request is listed in each endpoint's reference. You can find your application ID and API key in the [Algolia dashboard](https://dashboard.algolia.com/account). ## Request format Depending on the endpoint, request bodies are either JSON objects or arrays of JSON objects, ## Parameters Parameters are passed as query parameters for GET and DELETE requests, and in the request body for POST and PUT requests. Query parameters must be [URL-encoded](https://developer.mozilla.org/en-US/docs/Glossary/Percent-encoding). Non-ASCII characters must be UTF-8 encoded. Plus characters (`+`) are interpreted as spaces. Arrays as query parameters must be one of: - A comma-separated string: `attributesToRetrieve=title,description` - A URL-encoded JSON array: `attributesToRetrieve=%5B%22title%22,%22description%22%D` ## Response status and errors The Search API returns JSON responses. Since JSON doesn't guarantee any specific ordering, don't rely on the order of attributes in the API response. Successful responses return a `2xx` status. Client errors return a `4xx` status. Server errors are indicated by a `5xx` status. Error responses have a `message` property with more information. ## Version The current version of the Search API is version 1, as indicated by the `/1/` in each endpoint's URL.
17+
* The Ingestion API lets you connect third-party services and platforms with Algolia and schedule tasks to ingest your data. The Ingestion API powers the no-code [data connectors](https://dashboard.algolia.com/connectors). ## Base URLs The base URLs for requests to the Ingestion API are: - `https://data.us.algolia.com` - `https://data.eu.algolia.com` Use the URL that matches your [analytics region](https://dashboard.algolia.com/account/infrastructure/analytics). **All requests must use HTTPS.** ## Authentication To authenticate your API requests, add these headers: - `x-algolia-application-id`. Your Algolia application ID. - `x-algolia-api-key`. An API key with the necessary permissions to make the request. The required access control list (ACL) to make a request is listed in each endpoint's reference. You can find your application ID and API key in the [Algolia dashboard](https://dashboard.algolia.com/account). ## Request format Request bodies must be JSON objects. ## Response status and errors Response bodies are JSON objects. Successful responses return a `2xx` status. Client errors return a `4xx` status. Server errors are indicated by a `5xx` status. Error responses have a `message` property with more information. ## Version The current version of the Ingestion API is version 1, as indicated by the `/1/` in each endpoint's URL.
1818
*
1919
* The version of the OpenAPI document: 1.0.0
2020
* Generated by: https://openapi-generator.tech
@@ -29,7 +29,7 @@
2929

3030
namespace Algolia\AlgoliaSearch;
3131

32-
use Algolia\AlgoliaSearch\Model\Search\ModelInterface;
32+
use Algolia\AlgoliaSearch\Model\Ingestion\ModelInterface;
3333
use DateTime;
3434
use GuzzleHttp\Psr7\Utils;
3535
use Psr\Http\Message\StreamInterface;

‎scripts/cts/testServer/replaceAllObjectsWithTransformation.ts‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ const raowtState: Record<
1212
copyCount: number;
1313
deleteCount: number;
1414
pushCount: number;
15+
getEventCount: number;
1516
tmpIndexName: string;
1617
waitTaskCount: number;
1718
waitingForFinalWaitTask: boolean;
@@ -42,6 +43,7 @@ function addRoutes(app: Express): void {
4243
raowtState[lang] = {
4344
copyCount: 0,
4445
pushCount: 0,
46+
getEventCount: 0,
4547
deleteCount: 1,
4648
waitTaskCount: 0,
4749
tmpIndexName: req.params.indexName,
@@ -69,6 +71,7 @@ function addRoutes(app: Express): void {
6971
raowtState[lang] = {
7072
copyCount: 1,
7173
pushCount: 0,
74+
getEventCount: 0,
7275
deleteCount: 0,
7376
waitTaskCount: 0,
7477
tmpIndexName: req.body.destination,
@@ -88,7 +91,8 @@ function addRoutes(app: Express): void {
8891
console.log(raowtState[lang]);
8992
expect(raowtState[lang]).to.deep.equal({
9093
copyCount: 2,
91-
pushCount: 10,
94+
pushCount: 4,
95+
getEventCount: 4,
9296
deleteCount: 0,
9397
waitTaskCount: 2,
9498
tmpIndexName: req.params.indexName,
@@ -119,17 +123,21 @@ function addRoutes(app: Express): void {
119123
expect(req.body.action === 'addObject').to.equal(true);
120124
expect(req.query.referenceIndexName === `cts_e2e_replace_all_objects_with_transformation_${lang}`).to.equal(true);
121125

122-
raowtState[lang].pushCount += req.body.records.length;
126+
raowtState[lang].pushCount++;
123127

124128
res.json({
125-
runID: `b1b7a982-524c-40d2-bb7f-48aab075abda`,
129+
runID: `b1b7a982-524c-40d2-bb7f-48aab075abda_${lang}`,
126130
eventID: `113b2068-6337-4c85-b5c2-e7b213d8292${raowtState[lang].pushCount}`,
127131
message: 'OK',
128132
createdAt: '2022-05-12T06:24:30.049Z',
129133
});
130134
});
131135

132136
app.get('/1/runs/:runID/events/:eventID', (req, res) => {
137+
const lang = req.params.runID.match(/^b1b7a982-524c-40d2-bb7f-48aab075abda_(.*)$/)?.[1] as string;
138+
139+
raowtState[lang].getEventCount++;
140+
133141
res.json({
134142
status: 'succeeded',
135143
eventID: req.params.eventID,
@@ -152,7 +160,8 @@ function addRoutes(app: Express): void {
152160
expect(req.params.taskID).to.equal('777');
153161
expect(raowtState[lang]).to.deep.equal({
154162
copyCount: 2,
155-
pushCount: 10,
163+
pushCount: 4,
164+
getEventCount: 4,
156165
deleteCount: 0,
157166
waitTaskCount: 3,
158167
tmpIndexName: req.params.indexName,

‎templates/go/ingestion_helpers.mustache‎

Lines changed: 38 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,11 @@ func (c *APIClient) ChunkedPush(indexName string, objects []map[string]any, acti
1616
batchSize: 1000,
1717
}
1818

19-
waitBatchSize := conf.batchSize / 10
20-
if waitBatchSize < 1 {
21-
waitBatchSize = conf.batchSize
22-
}
19+
offset := 0
20+
waitBatchSize := conf.batchSize / 10
21+
if waitBatchSize < 1 {
22+
waitBatchSize = conf.batchSize
23+
}
2324

2425
for _, opt := range opts {
2526
opt.apply(&conf)
@@ -65,39 +66,41 @@ func (c *APIClient) ChunkedPush(indexName string, objects []map[string]any, acti
6566
}
6667

6768

68-
if conf.waitForTasks && (len(responses) % waitBatchSize == 0 || i == len(objects)-1) {
69-
var waitableResponses []WatchResponse
69+
if conf.waitForTasks && len(responses) > 0 && (len(responses)%waitBatchSize == 0 || i == len(objects)-1) {
70+
var waitableResponses []WatchResponse
7071
71-
if len(responses) > waitBatchSize {
72-
waitableResponses = responses[len(responses)-waitBatchSize:]
73-
} else {
74-
waitableResponses = responses
75-
}
72+
if len(responses) > offset+waitBatchSize {
73+
waitableResponses = responses[offset:waitBatchSize]
74+
} else {
75+
waitableResponses = responses[offset:]
76+
}
7677

77-
for _, resp := range waitableResponses {
78-
_, err := CreateIterable( //nolint:wrapcheck
79-
func(*Event, error) (*Event, error) {
80-
if resp.EventID == nil {
81-
return nil, reportError("received unexpected response from the push endpoint, eventID must not be undefined")
82-
}
83-
84-
return c.GetEvent(c.NewApiGetEventRequest(resp.RunID, *resp.EventID))
85-
},
86-
func(response *Event, err error) (bool, error) {
87-
var apiErr *APIError
88-
if errors.As(err, &apiErr) {
89-
return apiErr.Status != 404, nil
90-
}
91-
92-
return true, err
93-
},
94-
WithTimeout(func(count int) time.Duration { return time.Duration(min(500*count, 5000)) * time.Millisecond }), WithMaxRetries(50),
95-
)
96-
if err != nil {
97-
return nil, err
98-
}
99-
}
100-
}
78+
for _, resp := range waitableResponses {
79+
_, err := CreateIterable(
80+
func(*Event, error) (*Event, error) {
81+
if resp.EventID == nil {
82+
return nil, reportError("received unexpected response from the push endpoint, eventID must not be undefined")
83+
}
84+
85+
return c.GetEvent(c.NewApiGetEventRequest(resp.RunID, *resp.EventID))
86+
},
87+
func(response *Event, err error) (bool, error) {
88+
var apiErr *APIError
89+
if errors.As(err, &apiErr) {
90+
return apiErr.Status != 404, nil
91+
}
92+
93+
return true, err
94+
},
95+
WithTimeout(func(count int) time.Duration { return time.Duration(min(500*count, 5000)) * time.Millisecond }), WithMaxRetries(50),
96+
)
97+
if err != nil {
98+
return nil, err
99+
}
100+
}
101+
102+
offset += waitBatchSize
103+
}
101104
}
102105

103106
return responses, nil

‎templates/php/api.mustache‎

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -337,26 +337,26 @@ use Algolia\AlgoliaSearch\Exceptions\NotFoundException;
337337
$responses = [];
338338
$records = [];
339339
$count = 0;
340-
$waitBatchSize = (int)($batchSize / 10);
340+
$offset = 0;
341+
$waitBatchSize = (int) ($batchSize / 10);
341342
if ($waitBatchSize < 1) {
342343
$waitBatchSize = $batchSize;
343344
}
344345

345346
foreach ($objects as $object) {
346347
$records[] = $object;
347348
$ok = false;
349+
++$count;
348350
349-
if (sizeof($records) === $batchSize || $count === sizeof($objects) - 1) {
351+
if (sizeof($records) === $batchSize || $count === sizeof($objects)) {
350352
$responses[] = $this->push($indexName, ['action' => $action, 'records' => $records], false, $referenceIndexName, $requestOptions);
351353
$records = [];
352354
}
353355

354-
++$count;
355-
356-
if ($waitForTasks && (0 === count($responses) % $waitBatchSize || $count === sizeof($objects) - 1)) {
356+
if ($waitForTasks && !empty($responses) && (0 === sizeof($responses) % $waitBatchSize || $count === sizeof($objects))) {
357357
$timeoutCalculation = 'Algolia\AlgoliaSearch\Support\Helpers::linearTimeout';
358358
359-
foreach (array_slice($responses, -$waitBatchSize) as $response) {
359+
foreach (array_slice($responses, $offset, $waitBatchSize) as $response) {
360360
$retry = 0;
361361
362362
while ($retry < 50) {
@@ -380,6 +380,7 @@ use Algolia\AlgoliaSearch\Exceptions\NotFoundException;
380380
throw new ExceededRetriesException('Maximum number of retries (50) exceeded.');
381381
}
382382
}
383+
$offset = $offset + $waitBatchSize;
383384
}
384385
}
385386

‎tests/CTS/client/search/replaceAllObjectsWithTransformation.json‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -74,26 +74,26 @@
7474
},
7575
"watchResponses": [
7676
{
77-
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda",
78-
"eventID": "113b2068-6337-4c85-b5c2-e7b213d82923",
77+
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda_${{language}}",
78+
"eventID": "113b2068-6337-4c85-b5c2-e7b213d82921",
7979
"message": "OK",
8080
"createdAt": "2022-05-12T06:24:30.049Z"
8181
},
8282
{
83-
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda",
84-
"eventID": "113b2068-6337-4c85-b5c2-e7b213d82926",
83+
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda_${{language}}",
84+
"eventID": "113b2068-6337-4c85-b5c2-e7b213d82922",
8585
"message": "OK",
8686
"createdAt": "2022-05-12T06:24:30.049Z"
8787
},
8888
{
89-
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda",
90-
"eventID": "113b2068-6337-4c85-b5c2-e7b213d82929",
89+
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda_${{language}}",
90+
"eventID": "113b2068-6337-4c85-b5c2-e7b213d82923",
9191
"message": "OK",
9292
"createdAt": "2022-05-12T06:24:30.049Z"
9393
},
9494
{
95-
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda",
96-
"eventID": "113b2068-6337-4c85-b5c2-e7b213d829210",
95+
"runID": "b1b7a982-524c-40d2-bb7f-48aab075abda_${{language}}",
96+
"eventID": "113b2068-6337-4c85-b5c2-e7b213d82924",
9797
"message": "OK",
9898
"createdAt": "2022-05-12T06:24:30.049Z"
9999
}

‎tests/output/swift/Tests/Utils/Utils.swift‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,10 @@ public func XCTLenientAssertEqual(received: some Encodable, expected: String) {
6565
guard let unionizedData = try? JSONSerialization.data(withJSONObject: unionizedObject, options: .fragmentsAllowed),
6666
let unionizedJSON = unionizedData.jsonString?.data(using: .utf8),
6767
let unionizedString = String(data: unionizedJSON, encoding: .utf8),
68-
let expectedJSON = try? JSONSerialization.jsonObject(with: expected.data(using: .utf8)!, options: .fragmentsAllowed),
68+
let expectedJSON = try? JSONSerialization.jsonObject(
69+
with: expected.data(using: .utf8)!,
70+
options: .fragmentsAllowed
71+
),
6972
let expectedData = try? JSONSerialization.data(withJSONObject: expectedJSON, options: .fragmentsAllowed),
7073
let expectedMinifiedJSON = expectedData.jsonString?.data(using: .utf8),
7174
let expectedString = String(data: expectedMinifiedJSON, encoding: .utf8)

0 commit comments

Comments
 (0)