page_iterator_sync is added. [API-1761] (#1051)

Author: OzanCansel

Reference_Manual.md

@@ -146,6 +146,9 @@
          * [7.11.5 Read Row Metadata](#7115-read-row-metadata)
          * [7.11.6 Read Table Column Value](#7116-read-table-column-value)
          * [7.11.7 Error Handling](#7117-error-handling)
+         * [7.11.8 Iterators](#7118-iterators)
+            * [7.11.8.1 page_iterator](#71181-page_iterator)
+            * [7.11.8.2 page_iterator_sync](#71182-page_iterator)
    * [8. Development and Testing](#8-development-and-testing)
       * [8.1. Testing](#81-testing)
    * [9. Getting Help](#9-getting-help)
@@ -3680,13 +3683,64 @@ Exceptions which can be thrown by SQL API are stated at below:
   - `illegal_access` is thrown if page fetch operation is already progress. To prevent this wait for the `boost::future<sql_page>` which belongs to previous `next()` call.
 - `sql_result::iterator()` can throw two types of exceptions.
   - `illegal_state` is thrown if it is not an `SELECT` query or `sql_result::iterator()` is requested more than once.
+  - `sql_result::iterator()` throws `hazelcast_sql_exception` if it sql_result is already closed.
+- `sql_result::pbegin()` calls `sql_result::iterator()` method internally, so it throws same exceptions.
 - `sql_result::row_metadata()` can throw `illegal_state` exception if the result contains only update count.
 - `sql_page::sql_row::get_object(int)` can throw `index_out_of_bounds` exception if the index is out of range.
 - `sql_page::sql_row::get_object(std::string)` can throw `illegal_argument` exception if the column doesn't exist.
+- `sql_result::page_iterator_sync::operator++()` can throw `no_such_element` exception if the fetch operation is timed out.
+- `sql_result::page_iterator_sync::operator*()` can throw `no_such_element` exception if the iterator points to past-end element;
 
 In addition, any method which returns `boost::future<T>` can throw an exception.
 Unless otherwise is stated, `sql::hazelcast_sql_exception` is thrown.
 
+### 7.11.8 Iterators
+There are several iterators in SQL Api to satisfy different use cases. They are listed below:
+
+- `page_iterator` (Async page iterator)
+- `page_iterator_sync` (Sync page iterator)
+
+Reminder 1, only one type of iterator can be used. So it is suggested to choose appropriate one to fetch `SELECT` query results.
+
+Reminder 2, it is not possible to iterate from beginning to end more than once.
+
+Reminder 3, it is not possible to acquire iterators more than once, so consecutive calls to `sql_result::iterator()`, `sql_result::pbegin(std::chrono::milliseconds timeout)`, `sql_result::begin(std::chrono::milliseconds timeout)` methods will throw `illegal_state` exception.
+
+#### 7.11.8.1 page_iterator
+`page_iterator` is the basic and the most essential one. It supports fetching pages asynchronously by `next()` method which returns `boost::future<sql_page>`.
+Therefore, users are able to fetch `SELECT` query results page by page. It is acquired by calling `sql_result::iterator()` method.
+
+``` C++
+auto result = sql.execute("SELECT * FROM integers").get();
+
+for (auto itr = result->iterator(); itr.has_next();) {
+    auto page = itr.next().get();
+
+    std::cout << "There are " << page->row_count() << " rows the page."
+                << std::endl;
+
+    for (auto const& row : page->rows()) {
+        std::cout << "(" << row.get_object<int>(0) << ")" << std::endl;
+    }
+}
+```
+
+#### 7.11.8.2 page_iterator_sync
+`page_iterator_sync` is the second iterator which wraps `page_iterator` and serves it as a sync iterator. It allows users to iterator over pages.
+It is similar to native C++ iterators. So it can be used with `algorithm` header.
+It supports `operator*()`, `operator->()`, `operator++()` operators. Post increment operator is marked as `delete`.
+`operator++()`(pre-increment operator) fetches the next page in a blocking manner. `timeout` can be set. `operator++()` might throw `exception::no_such_element` exception if the fetch operation is timed out.
+`page_iterator_sync` is copyable but it is there only for convenience. Copy is shallow copy so copied instances should not be used simultaneously.
+This iterator is acquired by `sql_result::pbegin(std::chrono::milliseconds timeout)` and `sql_result::pend()`. `timeout` is defaulted to `std::chrono::milliseconds{ -1 }` which means wait forever.
+
+``` C++
+auto result = client.get_sql().execute("SELECT * FROM integers").get();
+
+std::vector<std::shared_ptr<sql_page>> pages;
+
+copy(result->pbegin(), result->pend(), back_inserter(pages));
+```
+
 # 8. Development and Testing
 
 Hazelcast C++ client is developed using C++. If you want to help with bug fixes, develop new features or

examples/sql/CMakeLists.txt

@@ -18,4 +18,5 @@ add_executable(sql_query_with_portable sql_query_with_portable.cpp)
 add_executable(sql_cancellation_example sql_cancellation_example.cpp)
 add_executable(sql_json_example sql_json_example.cpp)
 add_executable(sql_order_by_limit_offset sql_order_by_limit_offset.cpp)
+add_executable(sql_page_iterator_sync sql_page_iterator_sync.cpp)
 add_executable(sql_advanced_query_options sql_advanced_query_options.cpp)

examples/sql/sql_page_iterator_sync.cpp

@@ -0,0 +1,225 @@
+
+/*
+ * Copyright (c) 2008-2023, Hazelcast, Inc. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include <algorithm>
+#include <iterator>
+
+#include <boost/algorithm/string.hpp>
+
+#include <hazelcast/client/hazelcast_client.h>
+
+using hazelcast::client::hazelcast_client;
+
+void
+populate_map(hazelcast_client&);
+void
+create_mapping(hazelcast_client&);
+void
+for_loop(hazelcast_client&);
+void
+algorithm_copy(hazelcast_client&);
+void
+algorithm_filter(hazelcast_client&);
+void
+timeout(hazelcast_client&);
+
+/**
+ * Normally, cpp-client provides an async api for every features.
+ * But for sake of convenience and similar usages with native C++ iterators
+ * cpp-client provides sync page iterator. So `page_iterator_sync` is a blocking
+ * iterator. It wraps the `page_iterator` and allows to be used in sync
+ * manner. This example demonstrates how to use `page_iterator_sync` and what
+ * the use cases are.
+ */
+int
+main()
+{
+    auto hz = hazelcast::new_client().get();
+
+    // Preparation
+    populate_map(hz);
+    create_mapping(hz);
+
+    // Use cases, examples
+    for_loop(hz);
+    algorithm_copy(hz);
+    algorithm_filter(hz);
+    timeout(hz);
+
+    return 0;
+}
+
+void
+populate_map(hazelcast_client& client)
+{
+    // Populate a map before using it in sql.
+    auto map = client.get_map("integers").get();
+
+    for (int i = 0; i < 100; ++i) {
+        map->put(i, i).get();
+    }
+}
+
+void
+create_mapping(hazelcast_client& client)
+{
+    // Create mapping for the integers.
+    // This needs to be done only once per map.
+    // It is required to use a map in SQL query.
+    auto result = client.get_sql()
+                    .execute(R"(
+                CREATE OR REPLACE MAPPING integers
+                  TYPE IMap
+                    OPTIONS (
+                      'keyFormat' = 'int',
+                      'valueFormat' = 'int'
+                      )
+                    )")
+                    .get();
+}
+
+std::shared_ptr<hazelcast::client::sql::sql_result>
+select_numbers(hazelcast::client::hazelcast_client& client)
+{
+    using namespace hazelcast::client::sql;
+
+    sql_statement statement(client, "SELECT * FROM integers");
+
+    // Set cursor buffer size to 5
+    // So there will be 20 pages(100 / 5 = 20)
+    statement.cursor_buffer_size(5);
+
+    return client.get_sql().execute(statement).get();
+}
+
+void
+seperator(const std::string& text = std::string{})
+{
+    std::string output(60, '=');
+    boost::replace_first(output, std::string(text.size(), '='), text);
+
+    output = std::string(20, '=') + output;
+
+    std::cout << output << std::endl;
+}
+
+void
+for_loop(hazelcast_client& client)
+{
+    seperator("for_loop() - BEGIN");
+
+    auto result = select_numbers(client);
+
+    for (auto it = result->pbegin(); it != result->pend(); ++it) {
+        seperator();
+        for (const auto& row : it->rows()) {
+            std::cout << *row.get_object<int>(0);
+        }
+
+        std::cout << std::endl;
+    }
+
+    seperator("for_loop() - END");
+    seperator();
+}
+
+void
+algorithm_copy(hazelcast_client& client)
+{
+    using hazelcast::client::sql::sql_page;
+
+    seperator("algorithm_copy() - BEGIN");
+
+    auto result = select_numbers(client);
+
+    std::vector<std::shared_ptr<sql_page>> pages;
+
+    copy(result->pbegin(), result->pend(), back_inserter(pages));
+
+    std::vector<int> numbers;
+
+    for (const auto& page : pages) {
+        transform(
+          begin(page->rows()),
+          end(page->rows()),
+          back_inserter(numbers),
+          [](const sql_page::sql_row& row) { return *row.get_object<int>(0); });
+    }
+
+    sort(begin(numbers), end(numbers));
+    copy(begin(numbers),
+         end(numbers),
+         std::ostream_iterator<int>(std::cout, "\n"));
+
+    seperator("algorithm_copy - END");
+}
+
+void
+algorithm_filter(hazelcast_client& client)
+{
+    using hazelcast::client::sql::sql_page;
+
+    seperator("algorithm_filter - BEGIN");
+
+    auto result = select_numbers(client);
+
+    std::vector<std::shared_ptr<sql_page>> pages;
+
+    copy_if(result->pbegin(),
+            result->pend(),
+            back_inserter(pages),
+            [](const std::shared_ptr<sql_page>& p) {
+                // Filter out the pages which contains a number which is
+                // divisable by 20
+                return any_of(begin(p->rows()),
+                              end(p->rows()),
+                              [](const sql_page::sql_row& row) {
+                                  return *row.get_object<int>(0) % 20 == 0;
+                              });
+            });
+
+    for (const auto& page : pages) {
+        for (const sql_page::sql_row& row : page->rows()) {
+            std::cout << row.get_object<int>(0) << " ";
+        }
+
+        std::cout << std::endl;
+    }
+
+    seperator("algorithm_filter - END");
+}
+
+void
+timeout(hazelcast_client& client)
+{
+    seperator("timeout - BEGIN");
+
+    // `generate_stream(1)` generates a row per seconds, so it will guaranteed
+    // that it will timeout
+    auto result =
+      client.get_sql().execute("SELECT * FROM TABLE(generate_stream(1))").get();
+
+    auto it = result->pbegin(std::chrono::milliseconds{ 1 });
+
+    try {
+        ++it;
+        ++it;
+    } catch (hazelcast::client::exception::no_such_element& e) {
+        std::cout << "Timedout" << std::endl;
+    }
+
+    seperator("timeout - END");
+}